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Abstract 

This  rssssrch  invsstigstion  analyzes  on*  of  ths  first 
applications  of  ths  Critical  Buccass  Factor  (C8F)  Mat hod  at 
DoD  organization.  Tha  DoD  organization  that  tha  C3F  aathod 
mss  appliad  to  is  tha  Air  Forca  Systams  Command's  Foraign 
Tachnology  Division  (FTD) . 

Tha  application  Mas  accomplishad  using  tha  CSF  mathod 
davelopad  by  MIT's  Sloan  School  of  Managamant.  Tha  CSF 
mathod  usas  a  semi -structured  interview  and  opan  andad 
quastions  to  datarmina  tha  critical  information  naads  of  tha 
organization 's  top  managers.  Tha  CSF  mathod  overcomes  tha 
problem  of  providing  too  much,  and  vary  often  useless 
information  to  top  managers  that  other  methods  provide,  by 
determining  those  feM  areas  that  need  to  be  performed 
satisfactorily  in  order  to  accomplish  the  goals  of  tha 
manager/organization.  Tha  investigation  is  provided  Mith 
sufficient  detail  and  explanation  so  that  it  can  be  used  a 
guide  for  determining  tha  CSFs  of  other  DoD  organizations. 
Tha  results  of  tha  application  of  tha  CSF  mathod  at  FTD  show 
that  tha  CSF  mathod  can  be  successfully  appliad  to  a  DoD 
organization  for  determining  top  level  managers  information 
requirements.  Tha  results  of  this  investigation  indicate 
that  tha  CSF  mathod  is  useful  enough  to  be  considered  prior 
to  tha  development  of  any  DoD  organization's  top  level 
management  information  system. 


APPLICATION  OF 

THE  CRITICAL  8UCCE88  FACTOR  METHODOLOGY 
TO  DOD  ORGANIZATIONS 

I.  Introduction 

In  today's  world  of  business  operations,  it  is  becoming 
extremely  Important  for  organizations  to  incorporate  the 
right  business  practices  in  order  to  survive  in  the 
aggressive  competitive  nature  that  pervades  most  'ustries. 
In  order  to  gain  or  maintain  as  effective  and  /icient  a 
stature  as  possible,  organizations  of  all  varie*-!  are 
employing  computers  as  a  management  aid.  Data  from  all 
aspects  of  the  company  are  gathered  and  presented  in  some 
form  to  all  levels  of  management.  These  data  are  then 
processed  into  a  form  that  is  meaningful  to  the  user, 
becoming  information  that  can  be  useful  for  making  management 
decisions  <7i4). 

It  is  common  however,  that  the  top  managers '  information 
requirements  are  not  as  clearly  defined  as  they  are  for 
functional  managers  and  line  supervisors.  Hence,  these 
managers  are  often  presented  with  a  very  large  amount  of 
information.  From  this  inundation  of  information,  the  top 
managers  must  determine  what  is  relevant  to  that  particular 
position  and  what  is  not.  This  suggests  that  some  method  or 
technique  is  needed  that  will  define  the  top  manager's 
information  needs  and  alleviate  this  information  overload. 
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John  Rock art,  director  of  th#  Cantar  of  Information 
Syatams  Rasaarch  at  Tha  Sloan  School  of  Managamant, 
liasaachussatta  Inatituta  of  Tachnology  (MIT),  has  davalopad 
tha  Critical  Succaaa  Factor*  (CSF)  Mathod  of  datarmining  tha 
information  naada  of  top  managars.  CSF*  ara  tho*a  f aw  araas 
which,  if  parformad  satisfactorily,  will  ansura  successful 
performance  of  the  organization.  In  his  article  "Chief 
Executives  Define  Their  Own  Data  Needs”  (20) ,  Rockart 
explains  tha  advantages  and  disadvantages  of  several  methods 
used  to  provide  top  executives  with  tha  information  needed 
for  those  positions.  He  then  describes  the  CSF  method  which 
he  suggests  to  be  “highly  effective  in  helping  executives  to 
define  their  significant  information  needs"  (2C*e4-S5).  If 
this  CSF  method  is  as  effective  and  efficient  a  method  as 
described,  would  it  be  appropriate  to  apply  to  a  DoD 
organization? 

Sp  ecific  Problem 

Management  Information  Systems  (MIS)  are  intended  to 
provide  the  using  management  with  needed  information  about 
the  various  aspects  of  the  organization  from  which  decisions 
are  made  crucial  to  the  operation  of  the  firm.  As  will  be 
discussed  later,  this  is  not  necessarily  the  case  for 
existing  MISs.  The  problem  is  twofold!  1.  the  management 
does  not  us*  the  MIS,  and  2.  the  information  provided  by  the 
MIS  is  not  the  most  appropriate,  for  one  reason  or  another, 
for  helping  the  manager  make  a  decision  (8i29+,  4«34,  16i45, 
23 i 20 ) . 
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Evan  though  the  description  of  this  marginal  MIS 
situation  exists  for  many  industries,  it  is  not  limited  to 
the  private  sector  alone.  The  problem  exists  in  the  DoD 
Services  as  well.  This  is  evidenced  by  many  DoD  Service 
organization's  obsolesence  or  non-existence  of  Information 
Systems  (IS)  or  Decision  Support  Systems  (DS8>. 

These  systems  need  to  be  used  for  their  intended  purpose 
of  providing  needed  information  to  the  manager  as  an  aid  for 
making  decisions.  But  these  systems  need  to  be  designed  to 
accommodate  the  individual  manager.  "It  is  clear  that  a 
problem  exists  with  defining  exactly  what  data  the  chief 
executive  (or  any  other  general  manager)  needs"  (20i82). 

This  problem  exists  with  DoD  managers  as  it  does  with  other 
industry  managers  in  which  the  key  factors  or  critical  areas 
of  concern  to  a  manager  are  not  defined.  If  they  are  not 
defined,  that  information  is  more  difficult  to  obtain.  This 
situation  has  the  potential  to  reduce  the  effectiveness  of 
the  manager  and  therefore  the  performance  of  the 
organization. 

Scope  of  the  Research 

A  thorough  literature  search  has  failed  to  show  the  CSF 
method  as  having  been  applied  to  the  DoD.  With  the  limited 
time  available  for  completing  this  research,  it  was 
determined  that  the  CSF  method  could  be  applied  to  only  one 
major  DoD  organization  for  determining  the  specific 
application  of  the  C3F  method.  The  Foreign  Technology 
Division  (FTD)  within  the  Air  Force  Systems  Command  was 


selected  for  par-forming  this  rasaarch.  This  salaction  is 
basad  on  thair  racognition  of  tha  naad  for  dafining  tha 
information  requirements  of  tha  managars  Mi thin  tha 
organization,  and  on  thair  dasira  to  improve  tha  oxistlng  MIS 
at  FTD. 

Litaratura  Ravi aw 

Background.  Information  processing  technology  is 
quickly  becoming  a  standard  managamant  tool  that  has  uses 
from  word  processing,  to  decision  aids,  to  strategic 
management  diagnosis.  Tha  degree  to  which  information 
processors,  also  called  information  systems  (IS),  are  used  by 
managamant  varies  from  company  to  company,  and  from  manager 
to  manager  within  tha  companies.  These  managars  are  being 
provided  with  a  wealth  of  information  by  their  organizations' 
management  information  systems  (MI8) ,  but  much  of  this  in¬ 
formation  is  often  of  little  use  to  the  manager  because  it  is 
not  in  the  form  that  ha  needs. 

Decision  Support  Systems.  The  MIS  approach  of  the  past 
tended  to  give  the  manager  too  much  data  and  not  enough 
information  from  which  decisions  could  be  made  (29t20).  A 
Decision  Support  System  (D88)  improved  on  the  MIS  approach, 
and  is  designed  to  help  decision  makers  solve  their  company 
problems  by  providing  less  of  this  extraneous  information.  A 
DBS  can  help  the  manager's  decision  process  become  easier 
when  dealing  with  the  company's  semi -structured  and 
unstructured  problems  by  combining  the  use  of  computer  models 
with  human  analysis  (18i36-37).  It  is  a  commonly  used  term 


which  describe*  the  application  of  computer-based  systems  for 
providing  information  in  the  support  of  management  decision 
making  <13i84).  Much  that  is  now  written  and  said  about  the 
goals  of  D8S  "is  similar  to  those  things  that  were  espoused 
as  the  purposes  of  'management  information  systems'  (MIS)  a 
decade  or  so  ago”  <13iB4). 

How  must  a  DS8  be  better  than  a  MIS  in  order  to  be 

beneficial  to  management?  A  DSS  attempts  to  improve  on  the 

MIS  concept  by  being  different  in  three  wayst 

•DSS  must  be  effective  as  well  as  efficient.  The 
speed  in  sorting  through  millions  of  bits  of  infor¬ 
mation  is  not  as  important  as  the  'effect'  of 
the  system. 

•DSS  affects  all  managerial  levels  as  opposed  to 
MIS  which,  at  least,  was  perceived  as  a  service  to 
top  management  only. 

•DSS  is  less  structured  in  programs  and  procedures 
(29i  20) . 

With  the  improvement  in  the  DSS  approach  over  the  MIS 
approach  of  the  past,  the  organization  should  show 
improvement  in  performance,  which  should  thereby  help  insure 
the  future  success  of  that  organization  (8i39,  23 i 20-22) . 

What  Determines  Success?  Successful  operation  of  a 
firm  should  not  depend  on  the  information  system  alone.  It 
is  the  challenge  of  the  managers  to  use  the  information  that 
is  available.  In  most  cases,  "technology  is  not  the  stum¬ 
bling  block"  <17tl47).  The  reluctance  of  managers  to  proper — 
ly  employ  their  MIS  is  that  they  are  "disenchanted  by  their 
previous  exposure  to  ill  conceived  forays  into  management  in¬ 
formation  systems  and  word  processing,  land  doubt  that  their] 


workers  Mill  embrace  tha  naw  technology.  .  . "  (17ii47>.  Tha 
focus  of  many  of  thasa  managers  is  on  tha  problems  created  by 
computers  and  on  growing  budget  requests  rather  than  on  ways 
in  which  the  firm  can  make  use  of  this  technology  <14i2S). 

If  the  managers  believe  they  are  unable  to  control  the 
quality  of  the  MIS  output  of  the  firm,  they  are  unlikely  to 
rely  on  the  MIS  in  meeting  critical  goals  (4t34|  14i20-29| 

15i 30-31 j  22i3).  However,  to  be  successful  in  the  next 
decade,  executives  must  learn  to  deal  with  information 
technology,  as  well  as  devote  time  and  resources  to  the 
information  processing  activities  (14i25). 

The  next  logical  and  necessary  realization  of  MI8 
utilization  is  providing  the  manager  with  information 
designed  to  that  manager's  needs.  Once  information  needs  are 
accurately  identified,  "operational  procedures  and  computer 
software  can  be  developed  or  [existing  systems]  modified  to 
collect,  store  and  report  data  required  for  management 
purposes"  <23i303).  When  developing  these  information  needs, 
"managers  who  are  given  the  opportunity  to  obtain  essentially 
'free'  information  will  tend  to  request  more  than  they  need 
or  could  reasonably  use"  <lil49).  Becker  and  McClintock 
reviewed  research  findings  related  to  this  issue  and 
concluded  that  people  "appear  to  use  only  the  'key'  facts, 
yet  are  willing  to  pay  for  all  the  facts,  useful  and  useless" 
(3i473).  How  then  can  these  information  needs  be  determined, 
so  the  organization's  IS  can  support  them? 


Determining  Information  Nee ds 


Thar*  are  a  variety  of  ways  being  used  presently  that 
deal  with  developing  the  management-level  information  needs. 
Whether  these  methods  are  effective  in  achieving  Mhat  they 
purport  is  not  conclusive  from  the  reports  of  those  Mho  have 
employed  them. 

Nonetheless,  MhatevT  method  is  used  in  determining 
these  information  needs,  it  "should  be  able  to  1)  produce 
pTiodic  reports  tailored  to  the  unique  information  needs  of 
individual  manao*rs|  2)  produce,  on  request,  'demand 
reports, '  which  contain  more  detailed  information  than 
that  provided  in  routine  reports!  md  3)  support  ad  hoc 
inquiries  into  the  [firm's!  data  base  for  information 
associated  with  special  studies,  strategic  planning 
activities,  and  other  needs"  (23(303).  In  order  to  provide 
this  Information,  the  data  base  of  the  18  must  not  only 
contain  the  necessary  data  as  required  by  these  information 
requests,  but  the  means  must  exist  for  timely  updates  to  the 
data  base  so  the  information  produced  from  them  can  be 
relevant  and  currant  <23i303). 

The  by-product  technique  is  one  method  used  to  determine 
information  needs.  Homsvt,  with  this  method,  "little 
attention  is  actually  paid  to  the  real  information  needs  of 
the  chief  executive"  <20iB2).  The  information  to  be  supplied 
to  the  managers  is  determined  by  the  systems  analyst/ 
deslgnT.  The  system  used  to  supply  this  information  is 
usually  the  computT  that  is  dedicated  largely  to  the 
opTational  type  of  processes,  for  example  -  payrolls. 


accounting,  among  othmrs.  Thmaa  information  products  "arm 
mmsmntlally  by-products  of  a  particular  systam  dosignod 
primarily  to  perform  routine  paperwork  processing  ....  It 
leads,  to  Cal  welter  of  reports  .  .  .  Candl  has  the  paper¬ 
processing  tail  wagging  the  information  dog"  <20i82).  The 
only  advantage  of  this  frequently  used  method  is  it  doesn't 
take  up  managerial  time  <20t82,  23t304>. 

Another  method  is  the  Null  approach,  and  can  be 

described  by  the  attitude  thati 

Top  executives'  activities  are  dynamic  and  ever 
changing,  so  one  cannot  predetermine  exactly  what 
information  will  be  needed  to  deal  with  changing 
events  at  any  point  in  time.  The  executives, 
therefore,  are  and  must  be  dependent  on  future- 
oriented,  rapidly  assembled,  most  often  subjective, 
and  informal  information  delivered  by  word  of  mouth 
from  trusted  advisors  <20i82>. 

The  advantages  of  this  approach  contrasts  "the 
uselessness  of  the  reports  developed  under  the  by-product 
method"  (20t82).  Since  these  reports  will  not  be  used,  the 
manager  "relies  very  heavily  on  oral  communication"  (20i83). 
Roc kart  refers  to  Henry  flintzberg's  viewpoint  when  making  the 
point  that  "the  managers*  inputs  are  soft  and  speculative  - 
impressions  ...  hearsay,  gossip"  (20i83)  and  that  hard  data 
is  not  used.  However,  he  goes  on  to  say  that  there  are  data 
(formal  and  informal)  that  the  manager  needs  on  a  regular 
basis,  and  that  this  should  be  supplied,  which  this  null 
approach  does  not  do  (20i83). 

The  Key  Indicator  system  emphasizes  the  business  wide 
factors  in  providing  information.  There  are  three  concepts 
which  receive  this  attention. 


The  first  concept  is  ths  sslsction  of  s  sst  of  ksy 
indicators  of  ths  hsslth  of  ths  businsss.  Informa¬ 
tion  is  collsctsd  on  sach  of  thsss  indicators.  Ths 
sscond  concspt  is  exception  reporting  -  that  is, 
ths  ability  to  maks  availabls  to  ths  managsr,  if 
dssirsd,  only  thoss  indicators  Mhsrs  psrformancs  is 
significantly  diffsrsnt  (with  significancs  lsvsls 
nscsssarily  prsdsfinsd)  from  expected  rssults.  Ths 
SNScutiva  may  thus  pursus  all  ths  data  availabls  or 
focus  only  on  thoss  arsas  whsrs  psrformancs  is 
significantly  diffsrsnt  from  plannsd.  Ths  third 
concspt  is  ths  expanding  availabilty  of  bsttsr, 
chsapsr,  and  mors  flexible  visual  display 
tschniquss  <20i83). 

Ths  most  common  uss  of  this  systom  is  for  financial 
data.  MTha  r sport... is  svsr  changing,  but  its  orientation 
toward  'profit  and  loss'  and  'balance  sheet'  data,  as  wall  as 
ratios  drawn  from  thsss  financial  data  is  evident"  <20*83- 
64). 

An  approach  that  tries  to  improve  on  ths  existing  IS 
products  is  the  Total  Study  Process.  Ths  information 
requirements  of  the  managers  are  compared  to  that  which  they 
are  already  being  provided.  Deficiencies  are  identified  and 
are  satisfied  on  a  priority  basis  assigned  by  the  management 
(20i84).  "The  objectives  of  the  process  are  to  develop  an 
overall  understanding  of  the  business,  the  information 
necessary  to  manage  the  business,  and  the  existing 
information  systems"  <20iB4).  But  this  method  "is  expensive 
in  terms  of  manpower  and  Cisl  all-inclusive  in  scope" 

(20i84).  This  suggests  that  there  is  an  extremely  large 
amount  of  data  which  can  be  unwleldly  when  performing 
analyses. 

There  are  other  methods  possible  for  use  in  determining 
executive  level  information  needs.  Some  of  those  other 


mat hods,  as  wall  as  thosa  just  dascrlbed  ara  summarized  in 
Tabla  I.  This  tabla  lists  tha  methods'  names,  provides  a 
description  of  aach  method,  and  than  a  brief  advantage/disad¬ 
vantage  comparison. 

Tha  Critical  Success  Factor  Method.  The  C8F  method 
focuses  on  individual  managers  and  on  each  manager's  current 
information  needs.  It  provides  for  identifying  managerial 
information  needs  in  a  clear  and  meaningful  way.  Moreover, 
it  takes  into  consideration  the  fact  that  these  needs  will 
change  with  time  for  a  particular  manager. 

The  CSF  concept  was  developed  by  the  Sloan  School  of 
Management  at  the  Massachusetts  Institute  of  Technology  under 
the  direction  of  John  Rockart  "in  an  attempt  to  overcome  some 
of  the  shortcomings  of  the  Cotherl  approaches  discussed 
earlier  Cby  focusing!  on  individual  managmrs  and  on  each 
manager's  current  information  n mods  -  both  hard  and  soft” 

<20» 85) .  CSFs  arei 

the  limited  number  of  areas  in  which  satisfactory 
results  will  ensure  successful  competitive 
performance  for  the  individual,  department  or 
organization.  CSFs  are  the  few  key  areas  where 
'things  must  go  right'  for  the  business  to  flourish 
and  for  the  manager's  goals  to  be  attained  <6i7). 

This  concept  broadens  the  scope  of  information  systems  beyond 
the  more  commonly  regarded  computer  applications,  to  include 
all  kinds  of  information.  "The  challenge  is  to  develop  an 
effective  means  for  measuring  CSFs  and  monitoring  organiza¬ 
tional  or  individual  performance  with  information  systems 
through  creative  use  of  available  information  sources  and  the 
computer"  <12i 43-44). 


TABLE  I 


Common  Math  oris  for  Determining  Manager's  Information  Needs 

(23i 304) 

Hethod _ Description _ Advantaqes/diisdvantaqes _ 


Unstructured-intervieu 

Mthod 


Intrviemr  asks  eanager  to  lilt 
■pacific  types  of  intonation 
hi  or  ihi  needs  riportid  on 
routini  basis. 


Adv.i  Staple  to  uii. 

Diiidv.i  No  itructuri  tor  muring 
completeness  ind  conciseness  of 
information  specifications;  responses 
are  likely  to  bt  bimd  by  eanager  s 
ricwt  experiences. 


Incrinntil  (or 
prototype)  Mthod 


Manager  it  initially  ukid  to 
lilt  ainiaal  inforaation  nquirr 
Mnts.  Syitii  ii  deiignid  to 
providi  this  intonation.  Ov»r 
tin,  aanagir  ntinii  hit  or  hir 
intonation  Midi,  and  lyitw 
it  aodif led  accordingly. 


Adv.i  Dicnaiit  likelihood  ot  aanagir 
ipecitying  irrelevant  intonation; 
provides  tor  improving  coepletenass  ot 
intonation  ipecitications  over  tiee. 
Ditadv. i  Initial  ipecitication  ot  inter 
nation  needi  likely  to  be  incoaplete; 
•ay  involve  treguent  changes  in  infor¬ 
aation  systee  design. 


Group  process  eethod  Several  people  in  sieilar 

aanagerial  positions  reach 
consensus  on  types  ot  intoreation 
they  muld  like  reported  on 
routini  basis. 

one  eanager. 

Disadv.t  Intoreation  ipecitication  not 
tailored  to  needs  ot  the  individual 
eanager;  group  process  includes  no 
structure  ot  assuring  coepletenass  and 
conciseness  of  intonation  specifica¬ 
tions. 


Adv.i  Includes  ideas  of  several  eanagers 
with  sieilar  responsibilities,  and 
resulting  intonation  specifications 
likely  to  be  lore  coepl ete  than  those 
defined  with  unstructured  intrvim  of 


Current  report  analysis  SystM  analyst /designer  revient 
Mthod  reports  currently  provided  to 

eanager  and  eodifies  cyst  ms  as 
necessary  to  elieinate  duplica¬ 
tive  reporting,  consolidate 
reports  or  othmise  ieprove 
report  forests. 


Adv.i  Does  not  require  aanagerial  tiea 
in  intonation  requirceents  specifica¬ 
tion. 

Disadv.i  Does  not  consider  eanager  s 
vims  about  appropriateness  or  useful¬ 
ness  of  intonation  provided,  and  tends 
to  perpetuate  existing  systee  inade¬ 
quacies. 


Decision  analysis 
Mthod 


Manager  first  asked  to  identify 
•11  decisions  for  ehich  he  or 
she  is  responsible.  Manager 
and  analyst  aodel  each  decision 
and,  froe  eodels,  detrain* 
intonation  eanagr  nmds  to 
have. 


Adv.i  Provides  structure  (list  of 
decisions)  for  improving  completeness 
and  conciseness  of  intonation  specifi¬ 
cation;  guards  against  bias  in  responses 
froe  eanager 's  recent  expriences. 

Disadv.i  Cannot  assure  that  list  of 
decisions  an  coepl  ete;  does  not 
provide  for  identifying  inforMtion 
needed  for  eanageeent  control. 


CSF  Devtlopwnt.  Tha  CSF  concapt  is  based  on  an  ap¬ 
proach  that  -first  appeared  in  a  1961  Harvard  Business  Review 
article  by  D.  Ronald  Daniel  (9).  This  article  describes  the 
common  problem  most  companies  face  of  "inadequate  managerial 
information."  <9illl).  The  problem  is  not  a  lack  of 
information,  but  that  the  information  is  not  in  the  form  that 
is  useful  for  managerial  decisions.  Daniel  points  out  that 
an  HIS  must  "determine  the  factors  that  really  contribute  to 
the  competitive  success  in  the  particular  business  in 
question"  (9i 120) .  He  also  states  that  in  most  industries 
there  are  three  to  six  "success  factors"  that  Mill  determine 
that  particular  industry's  success. 

CSF  Areas.  The  areas  that  must  be  analyzed  to  determine 
an  organizations  CSFs  are  discussed  by  Daniel  (9) ,  Anthony 
(2),  and  Rockart  i20) .  Daniel  stresses  that  in  determining 
these  "success  factors,"  certain  areas  of  information 
generation  must  be  evaluated.  These  areas  of  information  are 
environmental,  competitive  and  internal.  Anthony  and  others 
use  three  areas  that  are  almost  the  same  as  those  discussed 
by  Daniel.  Anthony  adds  that  CSFs  must  be  tailored  to  the 
specific  company  and  to  the  individual  manager  (2i 148) . 
Rockart  (20)  uses  the  three  areas  of  Daniel 's  article  that 
focus  on  determining  success  factors,  then  through  analysis 
Rockart  and  the  Sloan  School  of  Management,  developed  four 
areas  that  must  be  analyzed  to  determine  CSFs.  These  four 
potential  sources  of  CSFs  as  provided  by  Rockart  (20i 86-87) 
arei 
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1)  The  structure  of  the  particular  industry |  this 
includes  ths  concerns  that  each  firm  of  the  industry 
have  in  common  due  to  the  nature  of  that  line  of 
business. 

2)  Competitive  strategyf  industry  position  and 
geographic  locationi  each  of  these  determine  the 
firm's  market  share. 

3)  Environmental  factors!  these  are  areas  which  are 
beyond  the  firm's  control,  but  must  be  dealt  with, 
such  as  business  or  economic  fluctuations  and 
trends. 

4)  Temporal  factors!  the  short  lived  areas  that  usually 
cannot  be  anticipated  very  far  in  advance  of  needing 
attention. 

The  above  factors  are  essentially  the  same  as  those 
first  presented  by  Daniel,  except  that  Rockart  and  the  Sloan 
School  of  Management  have  provided  an  expanded  explanation  of 

each  area. 

How  CSFs  are  Determined.  The  CSFs  of  an  individual  or 
an  organization  are  determined  through  the  use  of  a  two-phase 
interview  procedure  and  a  data  analysis  technique  (6x45-63). 
In  the  first  phase,  the  analyst  interviews  the  organization's 
executives  to  jointly  determine  the  managers'  CSFs.  Once 
this  phase  is  completed,  the  analyst  groups  and  combines  all 
of  the  Individual 's  CSFs  into  the  CSFs  and  information  needs 
of  the  organization,  followed  by  agreement  of  final  CSFs  by 
the  managers.  The  second  phase  of  the  interviews  is  then 
conducted  to  determine  the  measure  of  each  C8F,  and  the  data 
required  for  each  measure.  The  interview  process  helps  take 
the  manager's  implicit  information  requirements  and  makes 
them  explicit  in  the  form  of  three  to  eight  CSFs.  Rockart 
found  that  managers  had  a  clear  view  of  their  own  CSFs  and 
were  not  hesitant  in  identifying  those  CSFs  (19x12).  Once 


these  information  requirements  become  explicit,  the  MIS  can 
be  structured  to  provide  information  which  will  then  be  in  a 
form  that  is  useful  to  the  managers. 

Benefits  of  CSFs.  There  are  many  benefits  related  to 
the  use  of  the  CSF  method.  The  use  of  CSFs  helps  management 
focus  on  those  few  critical  areas  that  determine  the  success 
of  the  organization.  The  C8F  method  also  allows  executives 
to  determine  the  information  they  need  and  also  the  data  that 
their  MIS  must  use  to  produce  that  information  (21ill>.  CSFs 
also  help  remove,  or  at  least  reduce,  the  amount  of  useless 
information  managers  receive.  An  example  of  this  is 
portrayed  in  the  situation  facing  the  president  of  a  #60 
million  microwave  communication  sales  organization,  in  which 
he  was  receiving  97  reports  a  month,  but  few  provided 
anything  more  than  "score  keeping  data"  such  as  monthly 
profit  statements  <20i88).  Through  the  use  of  the  CSF 
method,  he  helped  determine  7  CSFs  that  provided  him  with  the 
information  he  needs  to  run  his  organization.  The  primary 
purpose  and  main  benefit  realized  of  CSFs  is  that  it  provides 
the  manager  with  more  time  to  deal  with  the  more  important 
functions  associated  with  his  or  her  position. 

Examples  of  CSF  Applied.  There  have  been  over  200 
companies  that  have  used  the  CSF  technique  (llt24).  The 
CSF  method  has  been  used  by  industries  such  as  high 
technology  manufacturing,  banking,  airline,  insurance, 
railway,  and  automobile. 

Bullen  (6i 22-25)  lists  the  current  CSFs  of  the 


automobile  industry  as  image,  quality  daalar  ayatam,  coat 


control,  and  aaating  anargy  standards.  Howavar ,  in  1961  tha 
automobile  C8Fa  includad  only  styling,  quality  daalar  ayatam, 
and  coat  control.  At  that  time,  tha  style  of  tha  car  had  a 
large  influence  on  car  purchases.  High  tail  fins  and  lots  of 
chroma,  among  other  features  Mara  prevalent  20  years  ago  and 
therefore  important  that  car  makers  appaal  to  tha  styling 
dasiras  of  tha  car  buyer.  Zt  Mas  also  important  that  daalar 
quality  Mas  high,  mostly  from  tha  vieM  that  tha  daalar  Mas 
the  direct  raprasantati va  of  tha  automakers.  This  Mould  have 
a  direct  bearing  on  current  car  purchases  as  Mall  as  future 
car  buys.  And  finally  cost  control  influenced  tha  auto 
industry  as  a  CSF,  since  profit  per  automobile  had  a  direct 
relationship  Mith  controlling  the  industry  costs.  Tha  more 
efficient  the  production  line,  the  higher  the  profits 
compared  to  other  automakers. 

In  1977,  after  the  government  required  the  industry  to 
meet  pollution  standards,  the  CSF  "meeting  energy  standards" 
mas  added.  It  can  be  easily  seen  that  this  Mould  be  an 
environmental  CSF.  Later,  in  the  1978  oil  price  surge, 
public  aMareness  of  the  limited  nature  of  fossil  fuels 
changed  the  industry  outlook  to  smaller,  more  fuel 
conservative  cars.  As  the  foreign  automobile  sales 
increased,  the  styling  CSF  mss  changed  to  image,  Mhich 
represents  efficiency,  reliability,  maintainability,  comfort 
and  patriotism  <6i22-23).  This  affected  the  vieM  of  cars 
from  flashy  style  features  to  less  frills,  aerodynamic  and 
more  efficient  "image"  that  currently  pervades  the  Industry. 
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Therefore,  the  original  list  of  auto  induatry  C8Fa  of 
styling,  quality  dealer  system  and  cost  control  changad,  to 
exchange  styling  faaturas  for  mora  imaga  faaturas  and  added 
energy  standards.  It  is  readily  evident  from  the  competitive 
nature  of  the  auto  industry  why  these  CSFs  are  at  work. 
Environmental,  temporal,  and  industry  factors  all  affect  the 
CSFs  of  the  industry  and  firms  within  the  industry.  Mhen 
these  factors  change,  as  they  have  in  the  auto  industry, 
firms  of  that  industry  will  have  to  accommodate  those  factors 
or  risk  losing  their  competitive  ability. 

In  another  example,  the  CSFs  of  the  computer  industry 
are  choice  of  the  market  niche,  technological  leadership, 
orderly  product  development,  service  and  stability,  and 
attraction/retention  of  quality  personnel  (6i25).  This 
industry  is  relatively  new  and  ever  changing.  Some  firms 
have  not  kept  up  with  certain  factors  and  have  had  to 
withdraw  from  the  market  from  lack  of  sales.  It  is  extremely 
important,  as  evident  from  these  examples,  that  CSFs  can 
determine  the  success  of  a  firm  in  its  industry. 

Comments.  Critical  success  factors  can  be  an  effective 
aid  in  the  management  of  organizations.  The  CSF  method 
enables  managers  and  organizations  to  define  their 
information  needs,  and  provides  the  needed  direction  for  MIS 
planners  to  produce  the  correct  information  for  the 
executives. 

There  is  very  little  documentation  on  the  subject  of 
either  CSF  in  theory  or  CSFs  having  been  applied.  This  is 


probably  because  tha  C8F  method  is  ralativaly  naw  and  only  a 
few  univarsitias  ara  taaching  or  researching  with  this 
particular  mathod.  It  is  possibla  that  tha  subjact  of  CSFs 
will  ba  discussad  mora  frequently  in  tha  litaratura  as  nor a 
casas  of  application  ara  documantad. 


Ralationship  Batwaan  Litaratura  and  tha  Problam 

Tha  litaratura  raviaw  prasantad  in  tha  background 
discussion  shows  that  tha  CSF  mathod  has  baen  usad  in  many 
commarcial  organizations,  but  has  yat  to  ba  appliad  to  tha 
DoD.  This  is  somawhat  surprising  since  both  commarcial  and 
DoD  organizations  require  that  information  flow  from  tha 
lower  levels  to  tha  higher  levels  of  tha  organization.  Since 
the  CSF  Method  has  proved  to  ba  an  affective  solution  for 
determining  tha  information  needs  of  high  level  managers  in 
commarcial  organizations,  CSFs  could  prove  to  be  a  solution 
for  determining  tha  information  needs  of  DoD  managers  as 
wel  1 . 

Justification 

There  have  bean  many  approaches  to  solving  tha  informa¬ 
tion  woes  of  managers  that  have  bean  mentioned  in  the 
previous  sections.  The  CSF  approach  has  bean  developed  to 
define  tha  critical  information  needs  of  those  responsible 
for  tha  operation  of  tha  organization.  The  81oan  School  of 
Management  has  demonstrated  tha  usefulness  and  successful 
application  of  CSFs  in  many  management  situations.  Tha 
various  reports  found  in  litaratura  described  earlier  have 
shown  positive  results  of  the  CSF  applications  involving  many 


different  industry  typss  and  a  variaty  of  organizational 
■factors.  Tha  adaptability  to  any  organization's 
circumstances  is  one  forte  of  the  C8F  method.  Although  it 
has  not  been  applied  to  a  DoD  organization,  the  strengths  and 
areas  of  focus  of  CSFs  are  well  suited  to  any  DoD  situation 
as  wel 1 . 

Objective 

As  stated  before,  it  appears  the  C8F  method  has  not  been 
applied  to  a  DoD  organization.  Vet,  with  the  importance  of 
national  defense,  effective  use  of  taxpayers'  money,  and  the 
voluminous  amount  of  data  and  reports,  required  or  not,  for 
the  day  to  day  activities  of  the  military's  managers,  it  is 
imperitive  to  equip  these  managers  with  an  effective  and 
efficient  way  of  accomplishing  their  job.  The  means  by  which 
this  is  done  should  not  require  much  time  or  resources,  so 
that  the  potential  benefits  of  employing  the  CSF  concept  are 
not  outweighed  by  the  costs  used  to  implement  them.  The  CSF 
method  can  do  just  that,  as  it  has  already  shown  the  managers 
of  other  industry  types  how  to  perform  more  effectively  and 
efficiently. 

The  objective  of  this  proposed  method  is  to  determine 
whether  or  not  the  CSF  method  will  work  in  a  DoD  environment. 
This  application  to  FTD  will  be  the  first  attempt  to  find  out 
the  appl icabi 1 i ty  of  CSF  to  a  DoD  organization  in  this 
manner.  From  the  results,  if  successful,  generalized 
recommendations  of  applying  CSFs  to  all  suited  DoD 
organizations  will  be  made.  The  ultimate  desire  is  to  reveal 


the  opportunity  of  reducing  the  plethora  of  paperwork  and 
waatad  of fort*  of  all  military  manager a,  ao  that  thay  may 
tha  little  time  available  to  them«  in  the  purauit  of  the 
orderly  accompl iehment  of  their  Joba. 


1 1 .  Methodoloi 


In  order  to  accomplish  ths  objsctivs  of  this  thesis,  as 
dsscribsd  in  Chaptsr  1,  an  iterative  procass  must  taka  placa. 
This  process  is  comprised  of  interviews  followed  by  analyses 
of  the  interview  results. 

Overview  of  C8F  Method 

The  CSF  method  involved  an  iterative  process  that 
includesi 

1.  An  individual  interview  with  the  top  managers  of  the 
organization.  This  group  of  managers  is  comprised  of  the 
commander  and  the  directorate  chiefs  of  the  organization. 

From  these  interviews,  the  broad  spectrum  of  Information 
requirements  will  be  established.  These  information 
requirements  will  be  in  the  form  of  each  managers  CSFs. 

2.  The  information  from  step  1  is  analyzed  for  common 
requirements  among  managers,  as  well  as  unique  information 
requirements  of  the  managers.  These  information  requirements 
are  combined  into  related  groups,  and  assigned  a  priority 
based  on  frequency  of  use  and  on  the  managers'  expressed 
needs. 

3.  Follow-up  interviews  are  conducted  with  the  same 
managers  from  the  initial  interviews.  The  first  cut  CSF  set 
derived  from  steps  1  and  2  are  presented  to  these  managers. 
During  these  follow-up  Interviews,  additional  information  is 
gathered  for  requirements  not  revealed  in  the  first 
interview. 
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4.  The  information  rmquirmmmnts  from  stop  3  arm 
final izod  into  thm  organization 's  and  individual  manager's 


critical  muccasm  factora. 

5.  Tha  findings  ara  rap or tad  to  tha  managara  involvad. 
This  raport  is  for  thair  uaa  and  submittal  to  tha  MIS 
managara  for  imp lamentation /mod if i cat ions  of  tha 
organization's  MIS  as  raquirad. 

The  CSF  process  as  described  above  is  provided  only  as 
an  overview  of  tha  process.  Tha  actual  process  as  applied  to 
FTD  is  described  in  more  detail  in  Chapters  III  and  IV. 


Particular  Method 

The  CSF  method  uses  personal  interviews  for  determining 
a  manager's  or  organization's  CBFs.  This  interview  technique 
uses  a  nonprobabilistic  purposive  sampling  technique,  and 
follows  a  structured  format  utilizing  open-ended  questions. 

The  basic  interview  technique  has  some  advantages  and 
disadvantages  as  discussed  by  Bailey  (3) i 

Advantages  of  the  interview  technique 

<1)  flexibility!  interviewer  can  probe  for  more 
specific  answers, 

(2)  control  over  environment!  interviewer  can 

standardize  the  environment  and  ensure  the 
respondent  has  privacy, 

(3)  spontaneity, 

(4)  respondent  alone  answers  questions  -  not  his 

staff , 

(5)  completeness!  interviewer  can  ensure  all 

questions  are  answered  completely, 

(6)  greater  complexity  of  questions  allowed. 

Disadvantages  of  the  interview  technique 

(1)  Cost, 

<2)  Time, 
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(3)  Interview  blast  intsrvlswsr  can  cause  srror  by 

misunderstanding  tha  raspondants  answer  or 
by  making  a  c lari cal  srror , 

(4)  No  opportunity  for  Interviewee  to  consult 

racords.  (3i 137-139) 

As  statad  aarliar,  tha  C8F  interview  mat hod  uses  a 
nonprobabi 1 istic  purposive  sampling  technique. 
Nonprobabilistic  sampling  uses  nonrandom  sampling.  This 
means  that  tha  sample  may  not  be  truly  raprasantati ve  of  tha 
total  population.  Tha  reason  nonprobabi 1 ity  sampling  is  used 
in  tha  CSF  method  is  that  only  certain  individuals  are 
selected  for  interviewing.  A  probabilistic  sample  would  not 
select  the  individuals  to  be  interviewed  to  obtain  a  random 
sample.  Therefore,  in  the  case  of  the  CSF  method,  since  the 
interviewers  need  to  control  who  is  interviewed,  there  is  no 
desire  to  randomly  interview  individuals  within  the 
organization. 

Again,  the  fact  that  only  certain  individuals  are 
interviewed  requires  the  CSF  method  to  use  a  purposive 
sampling  technique.  A  purposive  sample  is  one  that  involves 
a  "deliberate  effort  to  secure  a  sample  that  conforms  to  some 
predetermined  criteria"  (10ti77>.  The  criteria  used  in  the 
CSF  method  is  that  tha  interviewees  selected  are  the 
organization's  top  executives.  For  example,  if  the  CSFs  for 
an  organization  are  desired,  the  sample  consists  of  those 
individuals  who  are  in  the  best  position  to  provide  the 
necessary  information,  that  being  the  top  executives. 

A  structured  interview  format  is  used  to  provide 
consistent  interview  results.  This  facilitates  the  analysis 


of  the  interview  results  and  ihutn  that  all  of  tha 


question*  ara  askad  at  avary  interview  in  tha  mama  ordar. 

Tha  atructura  of  tha  CBF  interview  usad  at  FTD  is  dascribad 
in  Chaptar  III. 

Tha  C8F  structurad  intarviaw  format  utilizes  opan-andad 
intarviaw  questions.  "Opan-andad  questions  ara  usad  for  tha 
more  complex  questions  that  cannot  be  answered  in  a  few 
simple  categories  but  require  more  detail  and  discussion" 

(3i 107) .  Opan-andad  questions  allow  tha  respondent  to 
provide  his  thoughts  and  feelings  concerning  tha  topic  in 
question.  Bailey  (3)  provides  tha  following  concerning  opan- 
andad  intarviaw  questions! 

Advantages 

(1)  ara  usad  whan  tha  interviewer  wants  to  hear 
tha  respondents  view  on  a  particular  subject, 

(2)  allows  respondents  to  answer  adequately,  in 
as  much  detail  as  desired, 

(3)  ara  preferrable  on  complex  issues, 

(4)  allow  tha  respondent  to  be  creative  and  provide 
a  response  in  their  own  context. 

Di sad vantages 

(1)  provides  a  collection  of  worthless  or 
irrelevant  information, 

(2)  response  is  not  standardized  making  comparison 
more  difficult, 

(3)  require  superior  writing  skills  since  the 
interviewer  may  have  to  transcribe  a  lengthy 
statement.  <3« 106-107, 172) 

As  will  become  evident  later,  for  the  C8F  method,  the 

advantages  of  open-ended  questions  heavily  outweigh  their 

disadvantages. 

The  open-ended  interview  questions  that  are  used  in  the 
C8F  process  initially  focus  on  the  broad  scope  of  managers' 
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roles  in  the  organization,  and  prograsal valy  gat  mora  narrow 
in  scopa  to  ravaal  tha  mora  datailad  information 
raquiramants.  In  fact,  thasa  aama  basic  aat  of  quaations  can 
ba  uaad  r agar dl ass  of  tha  typa  of  organization  baing  daalt 
with.  Howavar,  it  takas  mora  than  just  a  list  of  quastions 
to  datarmina  an  individual's  or  organization's  C8Fs,  tha  mora 
datailad  and  ravaaling  Information  ralationships  can  only  ba 
collactad  from  an  understanding  of  tha  organization  and  from 
skillful  intarviaw  techniques. 

Tha  basic  open-ended  intarviaw  questions  and  related 
areas  that  ware  used  in  tha  initial  interviews  at  FTO,  as 
adapted  from  Bui lan  and  Rockarts  "A  Primer  on  Critical 
Success  Factors"  (6)  are  as  follows! 

Mission!  Describe  tha  key  missions  for  your 
organization. 

Role  in  Mission!  Describe  your  role  in  accomplishing 
each  of  these  missions. 

Soals  and  specific  tarqats  to  ba  reached  at  a  particular 
point  in  timei  For  each  mission  and  role,  what  are  your 
specific  goals? 

Measuring  goal  attainment!  How  do  you  datarmina  whether 
you  have  attained  each  goal? 

Critical  goals  of  individual  managers!  List  the  key 
areas  where  failure  to  perform  would  most  adversely  affect 
your  organization. 

Two  additional  quastions  helping  to  refine  most  critical 


[oalsi  In  which  area  would  you  most  hate  to  see  something  go 


wrong  and,  Suppose  you  have  Just  raturnad  to  your  job  attar 
30  days  of  laava  or  absancat  what  would  you  first  want  to 
know?  What  naxt?  (61 43-60) 

Tha  way  that  thasa  opan  andad  quastions  ara  utilizad  during 
tha  intarviaw  procass  for  devaloping  CSFs  is  dascribad  in 
Chaptar  III. 

Tha  usa  of  thasa  opan-andad  intarviaw  quastions  can 
raquira  tha  usa  of  probing  quastions.  Tha  probing  quastion 
or  follow-up  quastion  is  usad  in  ordar  to  hava  tha  raspondant 
explain  an  answer  more  fully.  Tha  probing  quastion  is  also 
usad  to  an sura  that  tha  topic  araa  tha  opan-andad  quastion 
addressed  is  covarad  in  sufficient  detail.  Examples  of 
probes  as  provided  by  Ballay  <3>  arai 

1.  Repeating  the  question.  This  is  dona  whenever 
tha  raspondant  hesitates  or  appears  not  to 
understand  tha  quastion.  With  lengthy 
questions  it  is  often  necessary  to  repeat  two 
or  thraa  times  before  the  raspondant  has  it 
clearly  enough  in  mind  to  begin  concentrating 
upon  an  answer. 

2.  Repeating  the  ansHtr.  This  type  of  neutral 
probe  can  be  used  by  tha  interviewer  who  is  not 
certain  that  he  or  she  understood  tha 
respondent's  answer  correctly.  Repetition  of 
tha  answer  can  correct  errors  and  assure  both 
respondent  and  interviewer  that  tha  answer  is 
recorded  correctly.  Repetition  also  gives  the 
respondent  an  opportunity  to  think  about  elabo¬ 
rating  it  further. 

3.  Indicating  understanding  and  interest .  The 
Interviewer's  Manual  (University  of  Michigan 
1969)  recommends  that  the  Interviewer  indicate 
that  he  or  she  heard  the  answer  and  approves  of 
it,  thus  stimulating  the  respondent  to 
continue. 

4.  Pause.  The  Manual  also  recommends  that  the 
interviewer  pause  and  say  nothing  if  the 
response  is  obviously  incomplete.  This 
indicates  that  the  interviewer  knows  the 
respondent  has  begun  to  answer  and  is  waiting 
for  him  or  her  to  finish. 

5.  A  neutral  question  or  coasant.  "How  do  you 


mean  that?"  or  "Tall  me  mora"  indicata  to 
tha  respondent  that  hi  a  or  har  antwar  is  on 
tha  right  track  but  that  mora  information  is 
dasirad.  <3i 173) 

Tha  probing  quastion  should  ba  as  nautral  as  possible  in 
order  to  avoid  biasing  the  respondent's  reply.  Probing 
questions  that  can  causa  problems  are  double  barrelled 
questions,  ambiguous  questions  and  leading  questions. 

Tha  double  barrelled  quastion  is  two  questions  in  one, 
and  should  be  avoided  when  asking  probing  questions.  It  can 
be  identified  when  the  words  "or"  and  "and"  are  contained  in 
the  question  <3i97).  When  the  double  barrelled  question  is 
used  as  a  probing  question,  the  respondent  may  become 
confused  as  to  which  question  to  answer.  The  respondent  may 
try  to  answer  one  or  both  questions  when  the  interviewer  is 
actually  expecting  the  opposite  response. 

Another  type  of  probing  question  to  avoid  is  one  that 
uses  ambiguous  words  or  phrases.  The  use  of  ambiguously 
worded  questions  can  cause  the  respondent  to  interpret  what 
the  questions  are  asking  and  possibly  answer  the  question  in 
that  light.  To  avoid  ambiguity  the  interviewer  should  pick 
his  words  carefully,  avoiding  slang  words  and  phrases.  In 
addition,  the  interviewer  should  assure  himself  that  the 
respondent  clearly  understands  the  question  when  it  is  asked. 
This  can  be  done  by  watching  for  a  puzzeled  look  in  the 
respondents  expression. 

Neither  the  double  barrelled  question  nor  the  ambiguous 
question  have  much  of  an  effect  on  the  outcome  of  the 
interview.  They  both  tend  to  cause  the  respondent  to  attempt 


to  anitwr  more  than  one  question  at  ones,  but  this  can  ba 
corractad  by  aaparatlng  or  rewording  tha  quastions. 

A  nor a  serious  problem  that  can  possibly  occur  while 
using  probing  quastions  is  biasing  tha  respondent  by  using 
leading  quastions.  Loading  quastions  are  those  that  attempt 
to  artificially  increase  tha  probability  of  a  particular 
response  (3» 102) .  Tha  use  of  leading  questions  can  be  a 
conscious  or  unconscious  act  to  try  to  get  the  respondent  to 
answer  in  a  certain  way.  In  either  case,  the  leading 
question  plants  a  thought  in  the  respondent's  mind  that  might 
not  have  been  there  before.  When  this  happens  the  respondent 
is  biased  and  his  response  may  be  different  than  if  the 
leading  question  had  not  been  used. 

The  biasing  that  can  occur  due  to  the  use  of  leading 
questions  is  due  to  something  known  as  anchoring.  Anchoring 
occurs  when  the  respondent  has  an  initial  response  and  keeps 
adjusting  that  response  until  the  final  answer  is  provided 
<24* IBS) .  It  is  similar  to  the  movement  of  a  boat  when  it  is 
anchored.  Once  the  anchor  is  placed  the  boat  can  only  move 
as  far  as  the  anchor  allows.  In  a  similar  manner,  the 
respondent  can  be  tied  to  the  original  thought.  Depending  on 
the  initial  placement  of  the  anchor,  the  final  answer  can  be 
fairly  close  to  the  respondents  true  belief  or  extremely 
different. 

As  mentioned  earlier,  the  C8F  method  requires  the  use  of 
open-ended  interview  questions  to  acquire  the  necessary 
information  from  managers  to  determine  their  CSFs.  In 
addition,  open-ended  questions  often  require  the  use  of 


probing  questions  to  fully  understand  ths  managers  response. 
The  interviewer  should  attempt  to  use  as  neutral  a  probing 
question  as  possible  to  avoid  the  problems  discussed  earlier 

Summary 

An  iterative  personal  interview  approach  is  used  for 
determining  top  managers'  Critical  Success  Factors.  These 
interviews  utilize  open-ended  questions  set  in  a  structured 
format  to  bring  out  the  managers'  information  requirements. 
Due  to  the  nature  of  open-ended  questions,  it  is  often 
necessary  to  use  probing  questions  to  follow-up  or  clarify 
the  respondent's  answer.  The  probing  questions  used  must  be 
as  neutral  as  possible  in  order  to  avoid  biasing  the 


respondent 


III.  Data  Gathering 


Introduction 

Tha  -following  description*  art  adaptations  o-f  the  C3F 
Primer  (6),  but  tailored  to  tha  specifics  of  a  DoD  type  of 
organization.  The  preparation,  preliminaries,  and  actual 
interview  descriptions  are  listed  in  order  of  desired 
occurrence,  and  felt  to  be  applicable  to  a  wide  variety  of 
DoD  organizational  situations.  Each  of  these  subsections  are 
followed  by  an  example  from  applying  the  CSF  method  to  FTD. 

As  will  be  explained  in  the  summary  and  conclusions  (Chapter 
V) ,  slight  variations  of  the  interview  process  can  and  should 
occur  in  order  to  adapt  to  the  peculiarities  of  the 
organization  in  question.  This  will  allow  maximum 
effectiveness  of  the  interview  and  overall  receptiveness  of 
the  C8F  methodology. 

As  mentioned  before,  the  method  of  applying  the  CSF 
technique  is  the  interview  process.  In  order  to  accomplish 
the  objective  of  determining  the  CSFs  for  an  organization, 
the  interview  must  be  conducted  so  as  to  take  very  little 
time  from  the  manager  being  Interviewed  while  extracting  all 
the  relevant  data  needed.  The  success  of  applying  CSFs  can 
hinge  on  the  success  of  the  Interview.  However,  the  success 
will  also  hinge  on  the  support  of  the  top  executives  in  the 
application  of  the  C8F  methodology.  A  step  by  step  procedure 
is  provided  below  which  is  designed  to  circumvent  many 
potential  problems  and  accomplish  the  objective  of 
determining  an  organization's  CSFs.  Note  the  emphasis  of 
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this  thesis  is  on  a  DoD  organization,  which  can  be  in 
reference  to  an  office,  SPO,  division,  and  others.  For  the 
purpose  of  this  thesis,  the  "organization"  is  the  basic 
reference  used,  and  is  meant  to  be  applied  to  any  DoD 
organizational  entity. 

Organizational  Details 

Before  embarking  on  a  campaign  to  "sell"  the  idea  of  the 
CSF  method  to  an  organization,  the  interviewers  should  know 
the  organization!  that  is  knowing  how  the  entire  business 
operations  are  conducted  currently  in  that  organization  (6i46- 
47).  It  is  recommended  that  a  team  of  2  or  3  interviewers  be 
used;  advantage  of  this  will  be  discussed  later.  This 

can  be  done  by  research  of  the  organization  through  its 
information  pamphlets,  brochures  and  other  publically 
available  sources  which  detail  the  missions,  organizational 
structure,  key  personnel,  corporate  plans,  and  similar 
descriptive  information.  The  information  gathered  from  these 
published  sources  may  or  may  not  provide  enough  detail  needed 
to  properly  understand  the  crucial  activities  that  occur  at 
the  higher  echelons  of  the  organization.  Other  details  can 
be  obtained  from  a  briefing  by  someone  within  the 
organization  knowledgable  of  these  activities.  This  can  even 
be  from  one  of  those  in  charge  of  the  IS  development. 

Applying  CSFs  to  an  organization  can  be  an  attractive 
management  aid  if  it  is  presented  to  the  executive  level 
properly.  Once  enough  details  about  the  organization  are 
gathered  by  the  interview  team,  it  may  be  necessary  to  "sell 


30 


the  CSF  methods  to  tha  upper  management.  It  is  imperative  to 
get  top  level  support  to  insure  at  least  adequate 
implementation  of  an  information  system.  Without  top  level 
support,  lower  level  managers  may  not  feel  implored  to  give 
so  freely  of  their  time  or  effort  in  allowing  the  C8F 
methodology  to  be  pursued.  Top  level  executive  support 
includes  not  only  portrayal  of  enthusiasm  and  direction  to 
the  immediate  directorate  level  chiefs,  but  a  willingness  to 
dedicate  an  adequate  portion  of  the  budget  to  the 
exploration,  definition,  development  and  implementation  of 
the  IS.  An  important  aspect  of  the  CSF  method  is  that  it  can 
define  the  IS  needs  quickly,  at  little  time  expense  to  the 
managers  involved,  and  with  relatively  small  dollar  amounts. 
The  little  expenditure  of  funds  and  effort  up  front,  when 
done  right,  can  not  only  save  many  dollars  later  on  for 
defining  the  right  system,  but  can  elicit  higher  level  use, 
and  therefore  success,  which  results  from  defining  the 
individual  manager's  particular  needs. 

Next,  the  top  executives  should  be  made  aware  of  the 
intent  of  CSFs  and  the  benefits  derived  from  the  CSF 
approach.  This  can  be  done  with  the  help  of  the 
organization's  IS  sponsors.  Initially,  a  letter  stating  the 
desireabi 1 ity  and  potential  benefits  of  CSFs  should  be  sent 
to  the  upper  management,  and  then  followed  by  an  optional 
briefing.  Depending  on  the  nature  of  these  individuals,  a 
briefing  may  be  the  desired  approach  to  discuss  the  approach 
and  benefits  of  CSFs,  and  allow  time  for  questions  about  the 
method.  This  requires  a  thorough  knowledge  of  the  CSF 


31 


process  as  Mall  aa  confidence  in  tha  ability  of  tha 
intarviawar  to  conduct  a  maaningful  intarviaw.  Tha 
davalopmant  of  tha  intarviaw  akilla  raquirad  ia  daacribad 
naxt . 

Bafora  going  on  however,  tha  avanta  to  thia  point  of  tha 
CSF  mat hod  that  occurrad  in  tha  FTO  axparianca  will  halp  show 
tha  potantial  flaxibility  of  tha  CSF  procaaa  in  ita  ability 
to  adapt  to  any  aituation  that  an  organization  might  offer. 

FTP  Exparianca 

Tha  Foraign  Tachnology  Diviaion  <FTP>  aatabliahad  an  MIS 
formulation  group  taakad  to  atudy  tha  axiating  IS  aituation 
that  had  bean  in  uaa  for  a  numbar  of  years,  and  now  proved 
inadequate  in  many  waya.  That  MIS  group  was  briefed  on  the 
CSF  method,  ita  banefita  and  atreaaad  tha  importance  of 
getting  top  level  aupport.  An  example  of  the  briefing  charts 
are  contained  in  Appendix  A. 

Tha  MIS  group  proceeded  than  to  develop  their  own 
veraion  of  that  briefing,  and  presented  it  to  the  commander. 
This  was  done  to  present  the  information  in  the  moat 
accommodating  way.  The  CSF  method  was  welcomed  as  having  the 
possiblity  of  alleviating  the  MIS  woes  that  many 
organizations  experience,  that  being  the  organization  trying 
to  accommodate  an  MIS  instead  of  tha  MIS  accommodating  tha 
organ! zation.  Tha  MIS  team  than  conducted  a  briefing  for  tha 


benefit  of  tha  intarviaw  team,  giving  further  details  of  tha 
FTP  mission,  and  how  each  directorate  and  division  fit  into 
tha  organization  to  accomplish  their  individual  objectives 


and  overall  mission.  In  ths  rsmaining  ssctions  of  this 
thesis,  thoss  msnsgsrs  interviewed  will  bo  referred  to  by  ths 
office  symbol  or  office  title  shown  in  Figure  1  below.  The 
organizational  chart  for  FTD  is  shown  in  Figure  2.  The  top 
managers  selected  to  be  interviewed  are  shown  in  mors  detail 
in  Figure  3. 
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Figure  1.  FTD  Office  Symbols/Titles 


The  FTD  commander  in  giving  his  full  support,  carried 
that  expentancy  of  support  over  to  the  directorate  chiefs, 
which  gave  an  overall  air  of  acceptance.  Once  the  individual 
managers  realized  that  they  will  actually  get  something 
useful  out  of  this  effort,  the  entire  C8F  application 
proceeded  very  wel 1 . 

Interview  Preparation 

A  go-ahead  to  apply  C8F  should  be  followed  by  a 


transmittal  letter  sent  to  the  top  managers  of  the 


Figure  2.  FTD  Organizational  Chart 


Figure  3.  Managerial  Chart  of  Interviewees 


organization  selected  for  tha  CSF  application  announcing  the 
CSF  af fort's  schadula  and  acknowledging  who  will  ba 
participating  in  tha  affort  <6i47-48>.  Thasa  managars 
includa  all  of  thosa  at  tha  top  level <s)  daamad  nacassary  by 
tha  MIS  support  group  and  tha  commandar.  Tha  1 attar  should 
indicata  tha  indorsement  of  tha  organization's  commandar,  or 
avan  ba  writtan  for  his/har  signature.  Tha  1 attar  should 
also  go  on  to  explain  tha  purpose  of  the  CSF  method,  tha 
intent  of  tha  interview  and  a  brief  outline  of  tha  interview 
proceedings.  Depending  on  tha  f lax ibility/avai lability  of 
tha  managars  to  ba  interviewed  a  tentative  schadula  could  ba 
established.  Schadula  no  more  than  three  interviews  par  day 
to  avoid  burn-out  of  tha  interview  team,  and  includa  other 
possible  blocks  of  time  in  which  the  team  is  available  to 
handle  tha  probable  schadula  changes.  It  is  recommended  that 
the  lower  laval  managars  ba  intarviawad  first.  "This 
provides  tha  ability  to  review,  in  preparation  for  interviews 
with  higher-level  managers,  tha  knowledge  gained  from  the  CSF 
interviews  of  managars  at  lower  levels"  (6i48).  This  also 
allows  tha  interviewer 's  confidence  laval  to  increase  before 
the  top  manager (s)  interviews  are  conducted,  which  invariably 
carry  a  larger  share  of  importance.  In  addition,  lower  level 
managers  typically  have  more  structured  responsibilities  than 
the  chief  executive,  and  this  also  makes  it  easier  to  define 
their  CSFs. 

In  order  to  avoid  possible  confusion  during  the 
interview  and  minimize  distractions  to  the  interviewee,  and 


thereby  raduca  tha  ef f acti vanaas  of  tha  attantion  givan  to 
tha  interviewee,  only  tha  individual  of  managamant  poaition 
should  ba  allowed  to  participata.  Tha  Mai 1  intandad  halp 
froa  Mi thin  that  manager's  organization  (even  tha  manager's 
closest  aid)  might  actually  ba  self -defeating,  and  raduca  tha 
affect ivanass  of  tha  interviaM. 

Tha  lnterviemee  Mill  benefit  from  any  background 
material  available  on  C8Fs  such  as  “Chief  Executives  Define 
Their  OMn  data  Needs"  <20)  and  could  ba  an  attachment  to  tha 
latter.  They  should  ba  asked  to  ba  ready  for  the  interview 
by  being  familiar  Mith  this  material.  They  should  also  ba 
cautioned  to  not  pra-datarmina  what  their  CBFs  are,  but 
simply  ba  able  to  answer  questions  dealing  specifically  Mith 
tha  managers  parvue.  Definitions  of  mission,  objective, 
goals  and  other  Important  terms  should  ba  included,  as  wall 
as  terms  that  nay  have  slightly  different  meanings  from 
individual  to  individual.  A  sample  letter  is  included  in 
Appendix  B.  The  attachment  to  that  latter  includes  a  listing 
of  the  tentative  interview  schedule  and  important  CBF  related 
definitions.  One  additional  item  in  the  letter  is  to 
indicate  not  only  who  is  participating  from  the  CSF  interview 
team  and  NIB  group,  but  when  the  interviews  are  scheduled  and 
where  tha  interviews  will  be  held.  This  should  be  a  location 
of  neutrallity,  not  in  the  interviewee's  office  if  it  can  be 
avoided.  The  surroundings  should  not  be  distracting  so  as  to 
reduce  the  effectiveness  of  the  interview.  Xf  this  neutral 
area  is  not  possible,  whatever  can  be  done  to  minimize 
distractions  (phone  calls,  meetings,  hallway  traffic  and  the 


like)  should  be  dons. 

Ths  sooner  ths  intsrviSMS  are  initiated,  ths  lsss  tims 
and  chance  of  ths  Millingness  of  support  from  these  managers 
to  Mane.  The  first  tMO  to  three  interviews  can  be  critical 
to  the  remaining  intervieMees '  responsiveness,  since  those 
intervleMed  Mill  most  likely  inform  the  remaining  managers  to 
be  intervieMed  hoM  the  intervieMS  so  far  have  turned  out. 
Therefore,  the  intervieMer  techniques  must  be  polished  and 
professional ,  or  else  the  effectiveness  of  the  CSF 
application  can  be  reduced  dramatically.  Before  the  actual 
intervieMS  are  conducted,  it  is  highly  recommended  that  the 
intevieM  questions  and  procedures  be  practiced,  preferably 
Mith  one  or  tMO  of  the  organization's  MIS  sponsor  members. 
This  not  only  Mill  add  confidence  to  the  interviewer,  but  it 
Mill  give  the  MIS  group  an  opportunity  to  make  helpful 
suggestions  on  how  to  approach  the  organization's  managers, 
and  hoM  to  deal  Mith  personalities  and  individual 
characteristics.  This  Mill  help  maximize  the  effectiveness 
of  the  interview,  and  also  allow  the  organization's  MIS  group 
to  become  familiar  with  the  C8F  interview  process  for  future 
applications  within  the  organization.  This  can  be  a  large 
benefit  since  inevitable  organizational  changes  in  personnel, 
structure,  or  changes  in  industry  can  cause  information  needs 
to  change,  but  these  changes  can  be  accommodated  by  updating 
the  CSFs  of  the  managers.  The  procedure  is  the  same  for  the 
practice  interviews  as  with  the  real  interviews,  and  the 
practice  interviews  should  be  treated  just  as  critical  in 


order  to  learn  how  to  adjust  to  unanticipated  responses , 
attitudes,  personalities  and  such  variables. 

The  inportance  of  the  interview  cannot  be  overstressed. 
The  Millingness  of  the  interviewee  to  reveal  the  lifeblood  of 
his/her  ieeediate  responsibilities  is  the  essence  of  the  C8F 
Method  which  cannot  afford  the  results  of  an  ineffective/ 
unaccomplished  interview.  The  interview  mechanics  reviewed 
in  Chapter  II  describe  the  various  factors  to  be  aware  of. 
Bullard  suggests  that  "from  all  acquired  knowledge,  Cof  the 
managers,  list  his/her!  probable  objectives,  goals,  CSF's  and 
measures"  <6* 49).  This  last  step  should  round  out  the 
interview  preparation. 

The  FTD  example  of  scheduling  had  to  remain  flexible. 
Initially  the  directors  were  notified  by  letter,  after  having 
been  briefed  of  the  tentative  schedule.  All  managers 
involved  were  at  the  directorate  level,  so  the  first 
interviews  were  scheduled  with  the  more  operational  side  of 
FTD,  support  directorates  were  next,  and  all  were  followed  by 
the  commander.  Availability  of  these  managers  was 
exceptional,  but  unforseen  circumstances  dictated  the 
allowance  for  alternate  days,  and  the  schedule  did  change  to 
accommodate  the  circumstances. 

The  interview  schedule  was  established  to  start  2  weeks 
from  the  briefing  to  the  MI8  group  so  as  to  allow  adequate 
time  to  prepare  for  the  interviews,  which  included  practice 
interviews  that  would  help  tailor  the  procedure  to  best 
accommodate  the  intricacies  of  the  FTD  organization. 

One  member  of  the  MIB  team  of  FTD  was  always  scheduled 


to  be  present  during  tha  interviews.  Thin  provad  to  ba  an 
assat  for  making  initial  introductions,  and  halpad  astablish 
guidelines  of  aacurity,  provida  rafrashmanta  and  othar  naada 

aa  thay  aroaa. 

Interview  Pral iminariaa 

Tha  intarviaw  should  bagin  by  briaf  nama  introductions 
(prafarably  by  tha  MIS  group  attandaa)  of  tha  participants  to 
add  to  tha  informal  atmosphara  that  is  daairad.  Ones  saatad 
and  ready,  ramind  tha  intarviawaa  of  tha  aganda,  purpose,  and 
nature  of  tha  interview.  If  tha  intarviaws  are  being 
conducted  by  a  team  of  2  or  3  members,  tha  main  interviewer 
should  sit  in  a  vantage  point  where  the  interviewee  is 
comfortably  close,  as  wall  as  having  relatively  easy 
visibility  to  at  least  one  member  of  tha  intarviaw  team.  The 
reason  for  this  will  become  evident  as  discussed  later.  The 
definitions  of  CSFs  objectives,  goals  and  measures  must  be 
wall  understood  by  tha  interviewee,  so  it  is  advisable  to 
revisit  them  at  this  point  to  insure  that  they  are  (6*50-52). 
It  should  help  to  give  the  interviewer  another  copy  of  the 
attachments  prior  to  the  interview  for  reference  during  the 
interview  in  the  event  there  is  still  a  tendency  to  confuse 
the  terms.  This  attachment  (or  point  paper)  should  not  be 
verbose,  but  a  concise,  understandable  listing  of  terms, 
definitions  and  intents.  An  example  is  provided  in  the 
attachment  to  the  transmittal  letter  in  Appendix  B. 

The  length  of  verbal  introduction  to  the  interview  that 
is  necessary  to  clearly  describe  this  point  paper  is 


dependent  on  the  interviewee's  knowledge  o-f  CBFs  as  provided 
in  the  material  contained  in  tha  transmittal  lattar  (6«S0- 
32).  Thasa  points  do  not  naad  to  ba  r aviawad  in  datail, 
howavar  thay  do  naad  to  ba  undarstood  before  going  on.  Mora 
tima  will  ba  raquirad  in  tha  avant  that  tha  managar  has  not 
raviawad  tha  matarial  as  requested. 

Tha  FTD  ax amp la  of  intarviaw  pral lminar ias  usad  tha 
appropriata  intarviaw  suggastions  as  axplainad  in  tha  CSF 
Primar  (6)  and  adaptad  thasa  to  tha  situation  anticipatad  in 
tha  FTO  organization.  Tha  intarviaw  point  papar  in  Appandix 
B  was  tha  guidalina  davalopad  for  that  purposa.  Tha  point  of 
raviawing  tha  objactiva  and  purposa  of  tha  intarviaw  includas 
an  axplanation  of  what  tha  CSFs  ara  and  why  tha  CSF  mathod  is 
baing  usad.  This  will  halp  sat  tha  staga  for  aliciting 
cooperation  from  tha  managar.  Tha  fact  is  raitaratad  to  tha 
managar  that  tha  results  of  this  study  will  result  in  an  IS 
designed  to  halp  him/her  do  thair  Job,  and  is  specifically 
tailored  to  thair  needs.  This  futhar  reinforces  tha  impetus 
for  the  managar  to  ba  vary  helpful  and  open  to  questions  that 
will  follow. 

This  review  of  tha  interviewer 's  objactiva  and  Intarviaw 
process  is  followed  by  a  statement  about  tha  informal  nature 
of  tha  interview.  Although  tha  intarviaw  is  structured,  any 
thoughts  on  tha  managers  mind  relevant  to  tha  issues  being 
pursued  must  ba  aired  unhindered  so  as  to  insure  tha  maximum 
potential  for  a  successful  intarviaw. 

Our  intarviaw  sessions  required  two  hours  with  each 
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interviewee.  Each  interviewee  was  expected  to  be  devoted  to 
the  -full  two  hours  in  the  event  the  -full  amount  of  time  was 
needed.  A  manager  that  has  his/her  attention  focused  on 
getting  back  to  the  job  or  some  other  engagement  will  not  be 
fully  effective  in  identifying  those  critical  areas  being 
sought  in  the  interview.  We  believe  that  two  hours  is 
representati ve  of  the  time  required  to  successfully  complete 
an  interview. 

Finally,  before  proceeding  directly  into  the  interview 
itself,  the  Key  Definitions  are  once  again  reviewed.  It  was 
not  uncommon  for  goals  and  objectives  to  have  alternate 
meanings  or  different  emphasis  than  that  listed  in  the  point 
paper.  It  is  necessary  to  remain  consistent  with  these 
definitions  adapted  for  the  interview  in  order  to  allow 
organization-wide  consistency,  and  allow  a  meaningful  follow- 
on  analysis  of  the  results. 

Interview 

As  with  the  point  paper,  the  questions  that  follow 
should  be  on  a  summary  paper  to  allow  smooth  continuity  of 
the  interview.  An  example  of  the  Question-Summary  paper  is 
included  in  Figure  4.  An  important  aspect  of  the  interview 
is  for  the  interviewer  to  maintain  eye  contact  and  portraying 
interest  in  what  the  interviewee  is  saying.  Therefore  this 
question-summary  paper  should  not  be  a  crutch  to  the 
interviewer  absorbing  undue  attention  from  the  interviewer, 
but  should  be  used  only  as  an  aid.  After  each  question,  some 
inconspicuous  marker  <e.g.  a  paperclip,  coffee  cup)  can  be 


used  to  provide  an  easy  eye  reference  when  needed.  After 
each  question  is  asked,  summarize  the  answer  to  ensure  that 
the  interviewer  understands  the  interviewee  response.  The 
worksheet  that  was  used  at  FTD  is  in  FiQure  3. 


*  Please  describe  the  key  missions  for  your  organization 

“Summarize  missions 

*  Please  describe  your  role  in  accomplishing  each  of  these 
missions. 

• 'Summarize  missions  fc  roles 

*  For  each  mission  and  role,  whate  are  your  specific  goals? 

**Are  there  any  goals  not  related  to  specific  mission 
and  role? 

■ ‘Summarize 

*  For  each  stated  goal,  how  do  you  determine  whether  you  have 
attained  the  goal? 

*  Please  list  the  key  areas  where  failure  to  perform  would 
most  adversely  affect  your  organization 

“In  which  area  would  you  most  hate  to  see  something 
go  wrong? 

• ‘Suppose  you  have  Just  returned  to  your  job  after  30 
days  leave  of  absence!  what  would  you  first  want  to 
know?  What  next? 

■‘Summarize  and  prioritize  C8Fs. 

*  For  each  of  these  critical  success  factorsi 

* *What  would  be  the  ideal  measure? 

“What  type  of  information  is  currently  available  to 
support  these  measures? 

■  Finally,  from  your  perspective,  what  are  the  Commanders' 
critical  success  factors? 


Figure  4.  Interview  Quest ion /Summary  Sheet 
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Ths  Worksheet  is  more  useful  if  it  is  legs!  size  paper 


turned  sideways  as  depicted  in  Figure  5  below.  The  area 
immediately  below  the  Missions/Roles/Goals/Measurement  work 
area  is  for  responses  to  the  CSF  prioriti zation  estions  and 
other  notes  as  needed  during  the  interview. 


Mi ssi ons 

Roles 

Goals 

Measurement 

Work  Area/  Question  Responses 

Figure  5.  Interview  Response  Worksheet 


The  interviewer  can  have  a  partner  or  team  responsible 
for  capturing  the  details  of  the  interview,  and  act  as  a  back 
up  to  the  interviewer  when  a  snag  is  reached.  This  is  where 
clever  signals  between  the  prime  interviewer  and  the  other 
team  members  may  prove  helpful.  When  any  problem  occurs  that 
requires  continued  attention  from  the  interview  team  such  as 
Interviewer  memory  lapse,  a  stagnant  moment  or  similar 
situations  that  distract  from  the  interview,  the  indiscreet 
request  for  help  can  allow  the  interview  to  get  quickly  back 
on  course.  Other  examples  of  back-up  can  be  asking  for 
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clarification  of  a  point  mada  by  the  interviewee,  halp  in 
clarifying  a  point  mada  by  tha  interviewer  by  using  altarnata 
phrasing  or  an  example,  or  as  an  aid  in  tha  summarizing  of 
aach  question's  rasponsa  bafora  moving  on  to  tha  next 
quastion.  Zn  fact,  it  is  possibla  to  altarnata  rolas  of 
intarviaixar,  or  rolas  of  asking  quastions  and  rola  of 
summarizing  responsas  batwaan  tha  taam  mambars.  This  can  be 
done  at  the  discretion  of  the  interview  taam  whan  it  is 
thought  to  be  an  acceptable  method.  This  will  vary  from  one 
interview  taam  and  organization  to  another. 

The  interview  proceeds  on  into  tha  structured  section  by 

first  asking  tha  manager  to  describe  tha  key  missions  of  that 

manager's  part  of  tha  organization  (6i52-53).  The  primer 

points  out  two  benefits  of  thist 

First,  it  is  an  easy  way  to  gat  tha  interviewee 
into  tha  process  and  to  start  him  talking.  Ha  is 
asked  to  discuss  that  which  ha  knows  bast,  his 
company  and  his  job.  Second,  however,  as  tha 
manager  discusses  his  job,  he  almost  always 
provides  clues  as  to  how  he  "views  tha  world". 

Is  ha  strategically  oriented?  Does  he  view  his  job 
as  one  which  is  sat  up  to  induce  change  into  tha 
organization,  or  is  he  a  "caretaker"  put  in  tha  job 
to  carry  out  tha  routines  which  have  been  developed 
by  past  managers?  <6i52) 

Again,  remember  to  allow  sufficient  time  for  the 
interviewee  to  answer,  and  keep  the  interviewer  on  the 
subject  as  much  as  possible.  Redirect  the  question  and 
clarify  it  if  the  response  is  not  in  line  with  the  Intent  of 
the  question.  If  the  response  to  the  mission  question  is 
simply  the  organization's  published  version  of  the  mission, 
stress  the  fact  that  tha  question  is  addressing  how  he/she 
personally  views  his/her  mission.  Once  the  mission  or 
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missions  ars  established,  summarizs  thsm  for  tha  managsr  and 
make  sura  that  this  agraas  with  what  ha/sha  intandad  to  say, 
and  maka  sura  thara  ara  no  othar  missions  left  uncover ad 
before  going  on  to  tha  next  topic. 

Next,  have  tha  manager  describe  his/her  role  in 
accomplishing  these  missions  <6i52-53).  It  is  helpful  to 
address  one  mission  at  a  time,  repeating  tha  question  for 
each  mission  area  to  elicit  tha  manager's  role  in  that 
mission.  Role  in  this  context  is  "what  docs  the  manager  do?" 
or  "what  is  the  manager's  job?”. 

The  same  interviewer  rules  apply  here  and  throughout  the 
interview.  Suggest  possible  areas  to  help  the  manager  answer 
a  question  whan  it  seems  hard  for  the  manager  to  grasp  an 
idea.  However,  don't  lead  the  interviewee,  or  put  words  in 
his/her  mouth,  but  from  the  interviewer 's  insight  and 
experience,  make  a  suggestion  which  can  spark  a  useful 
thought  on  the  subject. 

At  this  point  in  time  it  will  become  more  and  more 
evident  to  the  interview  team  as  to  the  nature  of  the  job  of 
the  manager.  Suggestions  and  clarifications  can  be  tempered 
to  follow  this  line  of  reasoning.  Is  the  manager  involved  in 
personnel  related  activities,  or  support,  or  operational 
activities?  The  more  interviews  conducted,  the  easier  it  is 
to  conduct  an  effective  and  efficient  interview. 

Once  the  first  mission  has  been  addressed,  move  on  to 
tha  next  mission.  If  the  role  is  complex,  it  is  suggested  to 
review  that  role  or  roles  of  that  mission  before  moving  on  to 


ths  nsxt  mission's  rols.  If  ths  rssponsss  srs  fairly  slmpla, 
move  on  co  ths  nsxt  mission's  rols,  in  succsssion,  until  thsy 
•rs  all  co vsr ad  sdsqustsly.  Thsn,  ones  this  has  bssn  dons,  a 
briaf  summarization  of  rolss  is  rscommsndsd  to  prspars  ths 
intsrviswss  for  furthsr  dstailing  of  ths  subjact. 

At  this  point,  a  closs  ays  on  tims  should  indicats  ths 
prograss  of  ths  intsrvisw.  Dspsnding  on  ths  complaxity  of 
rolss  and  managsr's  job  knowlsdgs,  45  minutss  to  1  hour 
should  havs  bssn  ussd  to  covsr  ths  missions  and  rolss.  Mors 
tims  than  this  Mill  probably  moan  an  axtandad  intar view  or 
folloM-up  ssssion  to  covsr  ths  crucial  arsas  of  ths 
intsrvisw.  Howavar,  this  is  not  a  hard  and  fast  ruls,  but  a 
good  indicator  from  which  ths  rsmaindsr  of  ths  intsrvisw  must 
bs  dstarminsd.  This  will  bs  discusssd  aftsr  ths  nsxt  arsa  of 
ths  intsrvisw)  ths  goals. 

In  a  similar  mannsr  as  with  ths  rolss,  ask  ths  managsr 
to  dsscriba  ths  spscific  goals  that  ha/sha  has  for  sach  rols 
(6»53-54).  If  it  is  nacsssary  to  clarify  again  ths 
distinction  batwssn  goals  and  objsctivss,  do  so  hsra.  Goals 
ars  spscific  targats  or  sfforts  that  havs  a  spscific  tims 
span  or  complation  data  assoclatsd  with  tham.  Ths  tims 
horizon  for  thass  goals  can  bs  that  which  is  most  moaningful 
for  him/hsr  <6i53).  "Most  managsrs  pick  a  ons-ysar  horizon) 
soma  provida  both  long  and  short  tsrm  goals.  Ths  choics  of 
ths  ti ms-span  and  naturs  of  ths  goals  sslsctad  ars  msaningful 
in  itsalf  sines  thsy  provida  furthsr  insight  into  ths  way  ths 
managsr  visws  his  job"  <6i53). 

Soma  managsrs  will  havs  goals  that  ars  not  formally 
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associated  with  ths  rols  of  the  manager,  but  one  that  is  a 
more  personal  goal.  An  example  o-f  this  is  "the  elevation  of 
Cthe  manager's]  sub-organization  higher  in  the  company  to 
give  it  (and  him)  more  power"  <6i54).  Since  both  the  formal 
goals  (those  associated  with  specific  roles)  and  informal 
goals  (such  as  those  just  mentioned)  are  used  to  derive  the 
CSFs,  it  can  be  seen  in  developing  the  manager's  CSFs  that 
"these  unstated  —  yet  very  important  —  informal  goals  Care] 
highly  useful  where  possible  to  bring  these  goals  to  the 
surface"  (6i54).  Once  these  formal  and  informal  goals  are 
listed  and  discussed,  they  too  are  summarized. 

The  next  steps  of  the  interview  process  are  designed  to 
develop  the  CSFs  from  the  manager's  goals  (6i54-58). 

Included  are  not  only  the  prioritizations  of  the  goals  and 
some  assessment  of  the  criticality  of  each  one,  but  an 
attempt  at  determining  the  best  measure  for  each  of  the 
goals.  The  means  of  measurement  are  extremely  useful  for  the 
IS  development  phase  which  immediately  follows  the  CSF 
application,  and  helps  determine  the  structure  of  the  data 
base  and  of  the  IS. 

In  determining  measures  for  the  goals,  it  is  recommended 
that  the  interview  be  structured  to  reserve  these  questions 
for  last,  and  if  time  left  in  the  ter view  is  short,  these 
questions  can  be  saved  for  the  second  interview.  So, 
"determining  measures  Cby  asking  measure  -  related  questions] 
does  not  need  to  be  carried  out  during  the  initial  CSF 
interview"  (6i39>.  Nonetheless,  the  process  of  determining 


measure*  is  dsscribsd  here,  but  ths  intsrvisw  schsduls  used 
is  left  to  tha  discration  of  tha  Interviewer. 

Tha  first  quastion  askad  that  is  usaful  in  achieving 
aaasuras  is,  "For  aach  statad  goal,  how  do  you  determine 
Mhathar  tha  goal  has  baen  attainad  or  not?"  Many  goals  will 
ba  aora  subjactiva  in  natura  and  difficult  to  assass.  Ona 
ax amp la  of  a  subjactiva  maasura  vary  common  at  FTD  is  thair 
raqulramant  for  good  intarparsonal  ralationships.  This  goal 
halpad  virtually  tha  antira  organization  oparata  mora 
afficiantly  to  soma  dagraa,  unf ortunataly ,  as  critical  as  it 
is,  thara  ara  no  convaniant  or  quantifiabla  maasuras  for  it. 
Thara  may  ba  attampts  mada  at  quantifying  this,  but  in 
contrast  to  a  dalivary  schadula  which  is  aasily  maasurad, 
intarparsonal  ralationships  ara  not  aasily  maasurad. 

Kowavar,  thasa  typas  of  maasuras  that  must  ba  considarad, 
diractly  or  indiractly,  and  maans  of  maasuramant  will  raquira 
imagination  on  tha  part  of  tha  intarviawar  and  tha 
intarviawaa. 

This  maasuramant  discussion  during  tha  intarviaw  and  tha 
othar  measuring  ralatad  questions  that  follow  can  ba  slightly 
lass  structured.  This  maans  that  soma  measures  may  ba 
difficult  to  determine,  but  "thara  is  no  danger  of  'leading* 
hare.  Tha  interviewer  may  suggest  maans  of  measurment  that 
tha  interviewee  did  not  think  of  but  likes  and  will  use" 
<6l39> . 

Once  these  goal  attainments  ara  discussed,  tha  next  area 
of  discussion  will  prioritize  tha  goals  and  will  determine 
tha  degree  to  which  they  ara  really  important  (6iS8-59>.  Tha 


reasoning  of  this  qusstioning  is  to  fores  ths  manager  to 
think  differently  about  these  goals,  but  are  “highly 
effective  in  assisting  managers  to  zero  in  on  the  few  most 
critical  areas  of  their  responsibility"  (6t55). 

The  initial  question  is  “which  of  the  key  areas  would  be 
most  adversely  affected  from  a  failure  to  perform  them?" 

This  obviously  has  the  manager  review  his/her  organization 
again  to  label  those  areas  that  must  be  done  in  order  to  keep 
the  organization  running.  This  in  effect  weeds  out  non- 
critical  goals,  and  places  emphasis  on  those  that  will  now  be 
called  CSFs. 

The  next  question  attempts  to  prioritize  the  more 
important  of  the  CSFs  and  put  it  on  the  top  of  the  list.  It 
is  "In  which  area  would  you  most  hate  to  see  something  go 
wrong?"  Some  managers  may  place  two  CSFs  there,  and  this  is 
acceptable,  since  equal  priority  CSFs  will  receive  equal 
attention  when  the  actual  IS  development  is  performed.  The 
idea  is  to  find  those  factors  that  are  most  critical  to  the 
manager  and  which  will  be  those  areas  that  truly  require  the 
support  from  the  IS. 

This  same  question  is  then  asked  in  a  more  thought 
provoking  way  as,  "Suppose  you  have  just  returned  to  your  job 
after  30  days  of  leave  or  absencei  what  would  you  first  want 
to  know?  What  Next?"  The  manager  may  say  that  that  long  an 
absence  would  never  happen,  but  again,  the  idea  is  to  be 
realistic  in  assessing  the  function  and  goals  of  the 
organ! zation.  Discussion  may  include  reasoning  why  these  are 
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thought  of  as  Important,  which  halpa  tha  manager  Justify  to 
him/herself  as  wall  as  to  tha  intarviawar  why  this  is  so. 

Tha  list  of  CSFs  at  this  point  should  ba  short  and 
prioritlzad,  and  should  bo  raviawad  for  tha  saka  of  tha 
managar  and  to  insura  tha  CSFs  ara  claarly  understood  by  tha 
interview  team. 

Measures  of  these  CSFs  are  now  attempted,  or  as 
mentioned  before,  attempted  during  tha  next  interview  <6«59- 
60).  Tha  first  of  these  questions  is  "What  would  ba  tha 
ideal  measure  for  those  CSFs?"  Each  C8F  is  addressed  one  at 
a  time  and  may  again  require  creative  thinking  for  the  more 
subjective  factors.  Follow  this  last  discussion  with  "What 
type  of  information  is  currently  available  to  support  these 
measures?" 

The  answer  to  the  last  question  along  with  the  responses 
of  the  CSFs  and  their  measures  is  the  objective  of  CSF 
methodology.  This  Information  is  what  the  MIS  support  group 
will  now  be  able  to  use  in  developing  the  IS.  Some  things  to 
do  and  remember  while  conducting  this  latter  section  of  the 
interview  when  specifically  addressing  C8Fs  aret 

1)  In  relation  to  the  CSF  classifications  of  temporal 
environmental,  and  so  forth,  "mentally  check  the  list  of  CSFs 
to  ensure  the  interviewee  has  not  focused  on  only  one  type  of 
C8F"  <6t56).  As  an  example,  make  sure  that  external  factors 
are  being  considered  as  well  as  the  immediate  organizational 
factors,  and  thereby  Insuring  that  a  complete  assessment  will 
be  obtained. 

2)  "'Aggregate"  CSFs  to  Insure  that  one  CSF  is  not 
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being  discussed  in  multiple  ways"  (6i57>.  This  is  similar  to 
the  first  point.  Instead  of  making  sure  all  CSF  areas  are 
being  addressed,  this  aggregation  checks  to  make  sure  one  CSF 
isn't  erroneously  being  addressed  in  these  other  CSF  areas. 

3>  As  described  before,  "check  to  ensure  that  all  CSFs 
are  being  elicited,  not  only  those  that  can  be  measured  with 
hard  data"  <6t57>. 

One  last  question  that  is  helpful  when  establishing  an 
organization 's  set  of  CSFs,  and  determining  where  the 
organisation's  emphases  are  is  "From  your  (the  manager's) 
perspective,  what  are  the  Commander's  CSFs?"  From  the  first 
manager  interviewed  on,  the  pieces  of  the  organization  start 
falling  together  into  a  more  complete  picture  with  the 
manager's  responses,  as  well  as  "arming"  the  Interview  team 
with  great  insights  of  the  organization's  purpose  prior  to 
interviewing  the  commander. 

Once  the  first  set  of  interviews  is  accomplished, 
usually  with  the  commander  being  the  last  one,  the  interview 
team  will  need  to  review  all  of  the  data  to  make  sure  the 
team  members  are  all  in  complete  agreement  on  the  CSF  lists 
and  meanings  for  each  manager.  These  lists  of  CSFs  are 
written  in  summary  fashion,  and  any  measures  for  them  are 
included.  This  forms  the  basis  of  the  second  interview. 

The  second  interview  is  very  informal  and  simply  a 
review  of  each  manager's  list  of  CSFs  to  make  sure  that  what 
he/she  meant  to  say  is  actually  reflected  in  the  team's 
summarized  version.  Any  changes  will  be  made  at  this  time. 
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If  the  first  intsrvisw  did  not  allow  for  ths 


determination  of  measures,  this  lins  of  questioning  is 
initiated  or  resumed,  until  each  CSF  has  a  form  of  measure. 

Lastly,  since  some  time  has  elapsed  between  this  and  the 
first  interview,  the  manager  may  have  additional  thoughts  on 
the  missions,  roles,  goals,  and  CSFs,  already  discussed. 

This  could  be  clarifying  words,  but  more  commonly  are 
additional  areas  not  covered  before  that  have  potential  for 
being  a  CSF  and/or  measurement.  These  are  appropriately 
discussed  here  until  the  manager  and  interviewer  are 
satisfied  with  the  results.  This  second  interview  usually 
culminates  the  CSF  interview  process,  except  in  some  cases 
for  an  additional  short  follow-up  session  only  as  needed 
before  presenting  the  whole  organization's  matrix  of  CSFs  and 
accompanying  analysis.  This  is  the  topic  of  Chapter  IV/. 

Summary 

Using  good  interview  techniques  and  instructions  set 
forth  thus  far  should  prove  very  useful  in  applying  the  CSF 
methodology.  The  method  is  designed  to  cover  the  garnet  of 
those  things  that  are  important  to  that  manager  at  that 
particular  time.  This  process  can  be  repeated  as  the 
organization  changes  with  its  environment,  or  as  managers 
change  as  is  so  common  in  the  DoD  Services. 

This  CSF  method  is  also  flexible  to  allow  changes  in 
questions  that  best  draw  out  the  managers  thoughts.  HThe 
interviewer  should  be  sure  that  he  has  'stretched'  the 
manager  as  far  as  possible  in  his  thinking  during  the 


interview"  (6i5B).  This  is  where  pursuing  alternate  vantage 
points  might  be  necessary  -for  giving  the  manager  a  clear 
grasp  of  the  idea  being  pursued  through  the  particular  line 
of  questioning.  As  is  necessary  for  successful  interviews 
"this  is  a  situation  in  which  one  must  be  extremely  sensitive 
to  the  problem  of  'leading  the  witness'.  The  interviewer 
must  walk  the  narrow  line  of  eliciting  information  without 
creating  the  answers"  (6i58).  Additionally,  the  interview 
structure  itself  can  be  flexible  to  accommodate  the  length 
time  of  the  interview  and  the  personalities  of  the  managers 
being  interviewed.  When  all  this  is  done,  the  interview  team 
is  ready  for  the  analysis  of  the  organization ' s  set  of  CSFs. 


IV.  Data  Analysis 


The  activity  that  follows  ths  intsrvisw  procsss  is  ths 
data  analysis  phasa.  At  this  tims  ths  CSF  intsrvisw  team 
should  havs  a  list  of  prioritized  C8Fs  for  sach  manager 
inter viewed.  Ths  list  of  C8Fs  should  havs  bssn  reviewed  with 
ths  appropriate  manager,  and  ones  final  agreement  is  reached, 
the  list  represents  that  manager's  current  and  complete  CSFs. 
The  analysis  of  the  interview  results  should  be  accomplished 
as  soon  as  possible  after  the  completion  of  the  interview 
sequence  to  avoid  the  possible  loss  of  pertinent  information. 

The  analysis  phase  of  the  CSF  application  allows  the 
interview  team  to  compare  the  CSFs  of  each  manager  in  order 
to  reveal  similarities  which  might  be  caused  by  the 
management  philosophy  of  the  organization  <6i61>.  In 
addition,  the  individual  manager's  CSFs  can  be  reviewed  to 
see  if  he/she  is  stressing  one  area  of  management  under  their 
control.  At  this  point  in  the  data  analysis  "it  is  highly 
desirable  to  prepare  a  written  version  of  the  CSFs  to  be 
reviewed  and  approved  by  the  interviewee.  In  addition  to 
ensuring  the  facts  are  correct,  this  step  may  elicit 
additional  information"  <6i61). 

Once  the  interview  team  is  assured  that  they  have  a 
reliable  CSF  list  for  each  manager,  they  can  now  proceed  in 
developing  a  matrix  of  the  CSFs.  This  CSF  matrix  will 
provide  the  organization's  information  system  planners  with 
an  idea  of  the  data  base(s)  required  to  provide  the  necessary 
information  from  the  top  managers'  CSFs. 
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In  developing  the  C8F  matrix  it  may  ba  nacassary  to 
group  and  combina  tha  individual  managara  CSFa.  This  is 
aspacially  trua  if  soma  of  tha  managara  hava  statad  aimilar 
or  ralatad  CSFa.  Howaver ,  tha  interview  taam  ahould  pay 
cloaa  attantion  to  tha  grouping  of  CSFa  ao  that  uniquely 
diffarant  CSFa  ara  not  loat  in  tha  procaaa. 

The  CSF  matrix  ia  conatructad  by  liating  tha  aggregated 
CSFa  along  tha  left  aide  of  the  matrix  and  the  managara 
interviewed  liatad  along  tha  top  of  tha  matrix.  Tha  matrix 
ie  then  filled  with  each  manager 'a  individually  prioritized 
ranking  of  CSFa  liatad  on  tha  matrix.  It  can  ba  beneficial 
to  organize  the  CSFa  on  the  matrix  by  one  of  the  four  sources 
of  CSFa  as  daacribed  in  Chapter  I,  thoae  of  industry 
structure,  strategy,  environmental  and  temporal. 

Catagorizing  the  CSFa  in  this  manner  will  assist  the 
information  system  planners  in  designing  the  information  data 
bases  required  to  accommodate  tha  information  needs  of  the 
top  managers.  These  categories  will  also  aid  the  top 
manager*  by  allowing  tham  to  determine  how  they  themselves 
are  grouping  their  information  needs  and  where  each 
individual  manager  is  placing  emphasis  on  the  overall  set  of 
information  requirements.  The  resulting  CSF  matrix  will  be 
similar  to  the  one  depicted  in  Appendix  C  as  adapted  from 
Figure  12  of  Bullen  and  Rockarts  "A  Primer  on  Critical 
Success  Factors"  <6i39-40). 

The  C8F  matrix  can  now  be  briefed  to  the  organization's 
top  managers  for  their  review. 


FTP  CSF  Data  Analysis 


The  analysis  of  tha  interview  rssults  from  ths 
Interviews  performed  at  FTP  did  not  occur  identically  with 
ths  procedure  described  above.  The  main  reason  -for  the 
change  in  procedure  was  tha  limited  time  available  to 
accomplish  the  entire  CSF  method  at  FTD.  In  fact,  FTP 
requested  the  CSF  method  be  applied  to  their  organization 
with  the  constraint  that  the  entire  process  be  accomplished 
within  30  days.  This  forced  the  scheduling  of  the  interviews 
of  the  top  managers  at  FTD  into  a  two  week  time  period.  The 
interviews  were  then  performed  over  a  two  week  period  and  due 
to  the  time  schedule  of  the  participants  involved,  the 
process  was  limited  to  only  one  interview  per  manager  of  no 
more  that  two  hours  in  length.  The  limited  amount  of  time 
allowed  for  these  interviews  forced  them  to  end  without  the 
development  of  the  Interviewee's  CSFs.  From  this  fact,  the 
CSFs  for  each  manager  had  to  be  determined  after  the 
conclusion  of  this  two  week  interview  process. 

The  actual  analysis  of  the  data  gathered  from  the  FTD 
interviews  started  with  comparison  of  the  interview  notes 
contained  on  the  Interview  worksheet  forms.  An  example  of 
the  interview  worksheet  for  FTD/SQ  is  shown  in  Figure  6.  The 
reason  for  starting  the  analysis  with  this  comparison  was  to 
ensure  that  each  interview  team  member  had  a  complete  and 
accurate  list  of  the  interviewees’  response  to  the  structured 
interview  questions.  This  was  accomplished  by  having  one 
team  member  read  the  reponses  out loud  that  he  had  written 
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Mission 


Roles 


Sosls 


1.  Receipt  and  processing 
of  data, 


1,  Revise  soee  (by  exception) .  1. 

Determine  resource  allocation 
and  deteraine  priorities. 

Plan  for  nee  capabilities. 


Anticipate  problems  further  in  advance 
and  take  corrective  action. 

Better  understand  implications  of 
Ieprove  Productivity  of  organization 
through  people  and  eachine. 

Develop  in  house  training  for  people. 
Keep  job  interesting  for  people. 


2.  Analyze  data  and  convert 
to  xritten  graphic 
fori. 


2.  Plan  for  nex  capabilities. 
Develop  nex  techniques  and 
tools. 

Mix  of  skills  and  training. 


2.  Upgrade  equipment  to  state  of  the  art. 
Earlier  integration  of  eulti  sensor 
inforeation. 


3.  Generate  report  on 
tasked  i teas  for  use. 


3.  Involved  in  late  reports  and  3, 
Management  by  exception. 

Periodic  reviex  to  Couander. 


Further  autoiate  the  reporting  process. 
Incorporate  electronic  tail. 


4.  Provide  feedback  to  4. 

collection  agency  or 
provide  specific  target 
requests. 

3.  Support  Interagency  5. 

xorking  groups. 


Coordination  for  nex  sensors  4. 
etc. 


Coordination  of  leetings.  5. 
Individual  assignment  for 
xorking  groups. 


Improve  timelines  of  collection. 
More  real  time  data. 


Reduce  the  investment  in  meeting 
support. 

Assign  most  qualified  individual. 


Failure  to  perform  mould  hurt  most  in;  Selecting  the  right  manager  for  the  job. 

Supervisory  relationships  betxeen  military  and  civilian. 

Internal  resources  available  to  support  nex  system. 

Mould  hate  to  see  go  xrong:  Working  relationships  of  people  in  directorate  xith  each  other  and 

xith  other  FTD  directorate. 


After  30  day  leave  xants  to  knox:  1st  -  status  of  operating  equipment. 

2nd  -  xhat  has  happened  to  funding  and  planning  schedule 
activities. 


Summary  of  main  Goals:  Ensuring  xe  have  out  year  funding. 

Reducing  the  training  time  of  nex  employees. 

Interagency  coordination  and  information  on  joint  procurement. 
Increase  the  productivity  of  generating  reports. 


Fig ure  6.  Interview  Worksheet  from  FTD/SQ  Interview 
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down  on  his  interview  worksheet.  By  raiding  through  each 
interviewee '»  rasponsaa  from  tha  interview,  tha  othar 
interview  taam  mambars  war*  givan  tha  opportunity  to  agraa 
with,  add  to,  or  discuss  a  differing  perception  of  what  they 
had  written  on  their  own  interview  workahaat.  Thia  waa  an 
important  atep  to  atart  with  ainca  it  happened  on  more  than 
one  occaaion  that  one  of  tha  intarviewara  had  miaunderatood 
or  even  missed  entirely  what  the  interviewee  had  said  in 
response  to  a  question. 

The  process  of  going  over  each  response  on  the  interview 
worksheet  ensured  that  not  only  was  tha  analysis  performed 
from  a  common  set  of  data,  but  the  most  information  as  was 
possible  had  been  extracted  from  the  data  gathered  from  each 
interview.  This  is  important  since  time  is  typically  a 
critical  resource  for  top  managers  of  an  organization,  so 
that  the  time  they  provided  to  the  interviews  had  to  be  used 
as  wisely  and  efficiently  as  possible.  This  was  especially 
important  in  the  FTD  instance  due  to  the  extremely  limited 
time  available  for  interviews. 

Following  agreement  on  the  contents  of  the  interview 
worksheets,  it  was  decided  that  each  member  of  the  interview 
team  should  individually  attempt  to  categorize  and  build  a 
matrix  of  C8Fs.  As  an  aid  in  categorizing  the  CSFs,  it  was 
noted  that  there  were  three  main  areas  where  the  commander 
and  directors  of  FTD  seemed  to  all  focus  on,  namely  external, 
internal  and  human  resources. 

The  interview  team  members  then  separately  developed  an 
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organizational  CSF  matrix.  This  separate  CSF  matrix 
development  allows  the  team  members  to  individually  reflect 
on  the  interviews  and  bring  out  their  own  thoughts  and  views 
on  the  outcome  of  those  interviews.  The  first  step  for  this 
was  to  list  the  managers'  goals  that  fit  into  each  main 
catagory  along  the  left  side  of  the  matrix.  The  next  s^ep 
was  to  list  the  individuals  interviewed  along  the  top  of  the 
matrix.  The  final  step  was  to  go  through  each  interview 
worksheet  and  check  off  the  appropriate  intersection  on  the 
matrix  where  the  managers  specifically  mentioned  the  goal. 

The  interview  team  then  reassembled  to  jointly  determine 
the  CSF  matrix  for  the  FTD  managers.  The  process  followed 
then  was  to  have  one  interview  team  member  read  through  the 
list  of  CSFs  under  each  of  the  three  main  areas  and  also  say 
which  directors  he  had  checked  for  specifically  mentioning 
that  CSF.  After  the  individual  did  this  for  one  CSF  the 
other  team  members  provided  feedback  relating  their  ideas  on 
what  was  presented  as  compared  to  the  CSF  matrix  that  they 
had  developed.  This  process  continued  through  the  remaining 
areas  until  agreement  was  reaehci. 

The  interview  team  then  looked  at  the  interviewee's 
responses  to  three  structured  interview  questions  to 
determine  the  first  and  second  area  of  concern  changing  the 
appropriate  checks  to  a  number  1  or  2.  These  three  questions 
were  i 

1.  List  the  key  areas  where  failure  to  perform  would 


most  sdvmrsmly  affect  your  organization? 

2.  In  which  arma  would  you  most  hat a  to  saa  somathing  go 
wrong? 

3.  Supposa  you  hava  Just  raturnad  to  your  Job  altar  30 
days  of  laava  or  absanca,  what  would  you  first  want  to 
know,  what  naxt? 

A  portion  of  tha  rasulting  CSF  matrix  for  FTD  is  shown  in 
Figura  7.  Tha  antira  CSF  matrix  is  included  in  Appendix  C. 
Note  that  there  are  items  marked  by  "x",  which  will  be 
discussed  later.  Tha  first  and  second  major  areas  of  concern 
are  denoted  by  an  "  <i>"  and  those  items  specifically 
mentioned  by  tha  manager  are  denoted  by  a  "xx". 

The  Interview  team  then  presented  the  briefing  on  the 
CSF  matrix  to  all  the  directors  interviewed  except  for  the 
command er.  This  allowed  the  directors  to  review  their 
individual  CSFs  with  only  their  peers  in  attendance,  and 
performed  as  a  replacement  for  the  second  interviews  normally 
performed  with  the  individual  managers.  This  group  approach 
of  reviewing  the  CSFs  was  a  result  of  the  severe  time 
limitation  mentioned  before.  The  preferred  approach  would 
still  have  been  to  review  the  CSF  lists  with  the  managers  on 
an  individual  basis. 

During  this  briefing,  the  directors  were  allowed  to  not 
only  validate  their  CSFs,  but  were  also  given  the  opportunity 
to  add  any  specific  area  not  discussed  in  the  prior  interview 
process.  The  items  that  the  managers  added  during  this 
session  are  denoted  by  an  MxH  on  the  CSF  matrix  shown  in 
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Figure  7.  C8F  Matrix  of  FTD 
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Foil  owing  the  briefing  to  the  director*  the  interview 
team  brio-fad  the  results  of  the  CSF  process,  including  the 


C8F  matrix,  shown  in  Appendix  C  to  the  FTD  commander  as  well 
as  the  directors  and  the  management  information  group. 

Summary 

The  data  analysis  phase  of  the  CSF  process  begins  with 
the  CSF  list  developed  for  each  manager  during  the  personal 
interviews.  Using  these  lists,  the  CSF  interview  team  groups 
and  combines  similar  CSFs  to  determine  common  information 
needs.  These  CSFs  are  then  listed  on  a  matrix  along  with  the 
managers  interviewed.  The  intersection  of  the  CSFs  and  the 
managers  specifically  mentioning  the  CSFs  is  marked  on  the 
matrix.  This  matrix  is  then  provided  to  the  organization's 
management  information  group  for  incorporation  into  the 
organization’s  information  system. 


Summary  and  Recommendations 


Tha  preceding  description  of  tha  CSF  method  and  actual 
application  to  FTD  encapsulate  the  currently  available 
literature  and  techniques  of  application.  The  major  focus  of 
the  CSF  method  is  to  provide  a  means  of  defining  the 
organization's  top  managers  information  needs.  From  this 
information,  the  managers  are  then  better  equiped  to  make 
decisions  that  really  affect  the  direction  of  the 
organization.  These  decisions  have  the  potential  to  directly 
affect  the  success  or  failure  of  the  operation  of  the  firm. 

The  recognition  of  top  management's  information 
requirements  attempts  to  be  an  improvement  over  other  methods 
of  Decision  Support  System  (CDS)  or  Management  Information 
Systems  (MIS)  that  may  only  satisfy  the  higher  detailed  needs 
of  lower  level  managers.  These  other  systems  tend  to  provide 
too  much  of  these  extraneous  lower  level  details  to  top 
management  from  which  upper  management  must  then  "weed" 
through  large  amounts  of  data  to  find  the  information  that 
is  really  pertinent. 

The  CSF  method  uses  the  interview  process  to  elicit 
these  critical  areas  of  concerns  from  upper  management.  This 
interview  is  structured  in  the  sense  that  a  format  is  used  to 
guide  the  data  gathering,  and  is  flexible  to  allow  for  proper 
clarif ication  or  amplification  as  needed.  From  the  hierarchy 
of  questions  directed  at  deriving  those  critical  areas  of 
concern  to  a  manager,  the  CSFs  are  established.  Those  CSFs 
are  then  identified  with  a  method  in  which  measurement  can  be 


quantified  for  use  in  a  computerized  data  system.  These  CSFs 
and  measures  are  the  basis  of  the  information  system 
specialists*  design  and  construction  of  the  data  base  to  be 
used  in  support  of  the  top  managers  decision  making. 

The  actual  application  of  the  CSF  method  to  FTD  was  a 
first  in  that  never  before  had  this  method  been  applied  to  a 
DoD  organization.  The  concern  was  of  the  rigid  non-profit 
structure  that  encompass  DoD  organizations  that  do  not  exist 
in  non-DoD  firms.  FTD  proved  to  be  a  prime  candidate  in  its 
representati veness  of  that  organization  to  most  organizations 
within  the  DoD,  and  of  their  own  recognition  of  the  need  for 
defining  top  management's  information  requirements  for 
their  crucial  role  in  the  world  of  global  military 
Intel  1 igence. 

The  application  of  CSF  proved  very  well  received  by  the 
entire  management,  and  very  well  suited  to  defining  the 
information  needs.  The  resulting  CSF  matrix  of  the  FTD 
organization  (Appendix  C)  is  the  information  structure  that 
is  currently  being  used  to  design  ,  develop  and  build  the 
information  system  to  be  used  at  FTD.  So  successful  was  the 
CAF  application  that  the  method  is  now  being  internally 
administered  to  the  lower  levels  of  management  from  which  the 
entire  organization  will  be  Integrated  from  the  information/ 
responsibility  point  of  view. 

Recommendat i ons 

The  CSF  method  proved  very  applicable  to  the  FTD 


organization,  and  therefore  any  DoD  organization  would  ba  a 
candidate  for  application.  With  tha  currant  emphasis  by  Air 
Force  System  Command  as  well  as  other  commands  to  employ 
state-of-the-art  computer  and  information  system  (IS) 
techniques,  the  time  seams  right  to  build  an  IS  around  the 
needs  of  the  organization.  The  potential  still  exists  to 
make  the  mistake  of  defining  information  needs  around  the 
computer  instead  of  building  the  IS  around  the  organization's 
needs.  The  CSF  method  has  bean  proven  very  effective  in  both 
DoD  and  non-DoD  applications.  Therefore,  it  is  recommended 
that,  before  an  IS  is  built  in  the  DoD,  the  CSF  method  be 
considered  first  for  applicability  in  providing  the  critical 
information  needs  of  the  organization's  upper  level  managers. 

A  follow-on  thesis  effort  is  suggested  to  covert 

1)  A  further  application  of  the  CSF  method  to  a 
larger  DoD  organization,  or  even  at  Command  level. 

2)  A  formal  "advert! zement"  of  the  CSF  method  at 
the  command  and/or  Air  Force  level  to  be 
incorporated  in  all  Air  Force  IS  applications  that 
warrant  it. 
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ADD  "IMAGE"  OF  QUALITY  AND  FUEL  EFFICIENCY 


COMPUTER  INDUSTRY  EXAMPLE: 
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MAIN  SOURCES  OF  CSFs: 


HOW  CSFs  ARE  APPLIED 


ADDITIONAL  FACT0RS7CLARIFYING  DISCUSSIONS 


SCHEDULE 


Appendix  B:  Sample  Transmittal  Letter  w/  Attachment 


DEPARTMENT  OF  THE  AIR  FORCE 
headquarters  foreign  technology  division  iafsci 
WRIGHT. PATTERSON  AIR  FORCE  BASE.  OHIO  AS43S 


1  0  APR  1933 

TQ 

»m-<*  XOFF 

mao  Critical  Success  Factors  Interview 


io  TQ  (Colonel  Wilson) 

1.  As  briefed  and  discussed  on  26  March  1984  at  the  MIS  Assessment  Team 
meeting  with  the  Commander,  Critical  Success  Factors  (CSF)  is  a  method 
for  determining  executive  information  needs.  Personal  interviews  identify 
factors  in  a  manager's  environment  which  must  be  accomplished  to  achieve 
desired  goals  and  objectives. 

2  AFIT  personnei  are  assisting  FTD  in  establishing  CSF's  for  an  FID  Executive 
Information  System.  Your  interview  is  scheduled  for  1300,  12  April  1984,  Room 
C204  and  will  proceed  along  the  following  general  outline: 

a.  Interviewer  -  open  the  interview 

b.  Interviewee  -  describe  his  mission  and  roles 

c.  Discuss  interviewee's  goals 

3.  Should  there  be  any  difficulty  in  maintaining  the  above  interview 
schedule,  please  contact  Mr.  J.  M.  Shawley  at  72348/225. 


Introduction 


Objective:  Assist  you  in  comprehending  the  significant  information 
that  is  needed  to  manage  your  organization. 

Process:  Critical  success  factor  study  as  briefed  by  Major  Rasch 

-  In-depth  interview  with  key  executives 

—  understand  missions  and  roles 

—  understand  goals  of  each  manager 

—  elicit  what  factors  determine  the  success 
of  achieving  each  goal 

—  make  an  initial  cut  at  establishing  measures 
for  each  success  factor,  and  associated 
information  needs 

-  Review  after  initial  analysis 

—  define  relationships  between  each 
individual's  CSFs 

—  important  to  remember  that  CSFs  are  particular 
areas  of  major  importance  to  a  particular 
manager  at  a  particular  point  in  time 


Nature  of  interview 

-  Informal,  but  structured 

-  Attempt  to  establish  mission,  roles,  goals,  success 
factors,  and  measures,  and  underlying  information  needs 

-  Last  approximately  2  hours 

Key  Definitions 

-  Critical  Success  Factors:  are  the  limited  number  of  areas  in 
which  satisfactory  results  will  ensure  successful  performance 
for  the  organization,  and  attainment  of  the  managers'  goals. 

-  Objectives :  are  general  statements  about  the  directions  of  the 
organization  without  stating  specific  targets  to  be  reached  at 
particular  points  in  time. 

-  Goals :  are  specific  targets  which  are  intended  to  be  reached 
at  a  given  point  in  time. 

-  Measures :  are  specific  standards  which  allow  the  calibration 
of  performance  for  each  goal.  They  may  be  "soft"  (subjective 
and  qualitative)  or  "hard"  (objective  and  quantitative). 


Atch 
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Appendix  C:  Data  Analysis  Results 


CRITICAL  SUCCESS  FACTOR  STUDY 
FOR 

FOREIGN  TECHNOLOGY  DIVISION 
AIR  FORCE  SYSTEMS  COMMAND 
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Critical  Success  Factor  (CSF)  Process 

•  Determine  CSFs  of  top  executives 

-  two  hour  structured  interviews 

•  Chart  individual  CSFs 

-  intersection  of  top  executives'  CSFs  represents 

the  set  of  CSFs  for  the  organization 

•  Derive  the  key  informational  data  base  requirements 

-  from  organizational  CSFs  with  objective  measures 

•  Integrate  informational  requirements  of  top  executives 

into  information  system  design  process 


Review  Individual  CSFs 


•  Objective 

-  validate  each  individuals'  CSFs 

—  clarify  subjective  interpretations  by 
interviewers 

--  add  any  specific  areas  not  discussed  during 
the  interview  sessions 

--  reminder:  focus  on  the  key  areas  of  activity  in 
which  favorable  results  are  absolutely  necessary 
for  a  particular  manager  to  attain  his/her 
goals . 

-  agree  on  the  set  of  CSFs  for  the  organization 


•  Observations 

-  found  substantial  agreement  among  individual  CSFs 

--  implies  that  FTD  is  an  excellently  managed 
organization 

—  striking  similiarities  exist  in  managerial 
viewpoints.  Specifically,  reliance  on 
good  interpersonal  relationships. 


FTD  Critical  Success  Factor  Study 
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Table  1 

Key:  (i)  -  denotes  first  and  second  major  areas  of  concern 
xx  -  indicates  that  CSF  was  specifically  mentioned 
during  the  interview  process 
x  -  indicates  that  CSF  was  identified  during 
1  focusing’  session 
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^0-fii48  250  APPLICATION  OF  THE  CRITICAL  SUCCESS  FACTOR  METHODOLOGY  2/2 
TO  DOD  ORGANIZATIONS)  AIR  FORCE  INST  OF  TECH 
HRIGHT-PATTERSON  AFB  OH  SCHOOL  OF  SYST.  . 

UNCLASSIFIED  D  P  CARROLL  ET  AL.  SEP  84  F/G  5/i  NL 


Key  Information  Data  Base  Requirements 

•  Define  information  data  base 

-  data  repositories  to  make  information  available  for 

recall  and  analysis 

-  limited  role  in  day-to-dgy  operations 

•  Key  systems  identified  from  organizational  CSFs 

-  Task  Monitoring  Subsystem 

—  from  receipt  of  tasks  through  completion 
--  maintain  history  of  tasks 

-  Planning,  Programming,  &  Budgeting  Monitoring 

Subsystem 

--  from  initial  plan  through  completion  of  project 

—  include  all  line-item  funding 

-  Human  Resources  Monitoring  Subsystem 

—  people  data  base;  including  skills,  assignments, 

promotion  data,  training  and  education, 
awards  and  incentives,  security,  incidents,  etc 

•  Limitations  of  such  systems 

-  dependent  on  data  collected  and  maintained 

-  cannot  answer  inferential  questions 

—  What  caused  a  late  report? 

-  cannot  establish  causality  relationships  among  data 

--  What  effect  does  PME  have  on  promotion? 


Relationship  of  Information  Data  Bases  to 
Organizational  Critical  Success  Factors 


1 

1 

Information  Data 

Base 

Organizational  CSFs 

{Task 

i Monitoring 

Ippbs 

{Monitoring 

{ Human 
{Resources 

ance  the  Image  of  FTD 
Provide  Quality  Service 
Create  Favorable  Impression 
Maintain  Technological 
Leadership  Role 

osition  the  Intelligence 
Function 

agement  of  Tasking 
Monitor  Status 
Assessment  of  Resources 
Establish  Priorities 

ductivity 

Enhance  Intra-Agency 
Cooperation 

Expand  Use  of  Management 
Analysis 

Increase  Automated  Support 


Suggested  Structure  of  Information  System 


Data  Dase 


Interface  Modules 


Decision  Level 


Need  to  establish  information  requirements  for  Division 
and  Eranch  level  managers. 

-  Apply  a  modified  CSF  method 

--  use  questionnaire  in  a  group  meeting. 

—  establish  managers  role  in  performing  mission 

—  define  specific  information  requirements  which 

support  those  roles 

-  Objective  is  to  link  MIS  to  managers’  needs 

Develop  decision  scenarios 

-  Identify  typical  questions  that  need  answers 

—  at  each  management  level 

-  Allows  validation  of  the  identified  information 

requirements . 

Design  Prototype  Systems 

-  Rapid  development 

-  Lowers  risk  of  developing  non-suppor ti ve  system 
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